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SUMMARY 


ABSTRACT 


In the case of Mexico, educational policies in general, and in particular, those aimed at the 
University subsystem, must be aimed at preparing the country and its inhabitants to respond 
to the new challenges posed by 21st century society, and among the latter is the need to 
improve the quality of education. The present work studies the theoretical foundations around 
the evaluation of the educational quality in the Public Universities of Mexico (UPM), using as 
a base the specialized literature in the reference topic. After a review of various evaluation 
perspectives, the document builds a methodological synthesis exercise on the evaluation 
process in Mexico, from which some critical elements of said scheme are established and, 
finally, an outline of a proposal is formulated. for the evaluation model of the PSUs, where 
it is suggested that evaluation must become an effective means for quality assurance and 
accountability. Thus, the current document highlights the consideration of the following 
formal foundations for the evaluation of educational quality in UPMs: 1) Quality assurance; 
2) accountability; and, 3) the establishment of an autonomous evaluation institution. Around 
this last point, the proposal that is configured in this essay points towards the formation of 
a State Evaluation Institution with administrative and financial autonomy, as well as political 
independence, which performs academic evaluation as an instrument of accountability. And 
the feasibility of the proposal is found in the same rendering of accounts, since it is, in itself, an 
option for a technological solution to the asymmetric information that exists between citizens 
and their representatives. 

KEYWORDS: Evaluation, Educational Evaluation, Educational Quality, Evaluation of 
University, Quality of University, Evaluation of Educational Quality, Public Universities, Mexico. 


INTRODUCTION 


In the last two decades, education in general, and University in particular, has 
undergone transformations in many parts of the world due to economic, political and social 
causes, which have translated for educational institutions into an increase in enrollment, the 
decrease in funding, the search for alternative sources to carry out their tasks, the change 
in the relationship between the university and the State, in addition to the concern of society 
and the government for the achievement of educational quality and for the investment made 
in the latter. The evaluation scheme, and specifically the evaluation of educational quality, 
arises in this context of changes that entails opportunities and challenges. 


In the case of Mexico, educational policies in general, and in particular, for the 
University subsystem, must be aimed at preparing the country and its inhabitants to respond 
to these challenges posed by 21st century society. Among the latter is the need to improve 
the quality of education, in consideration of the development of academic staff and the 
commitment of University institutions to respond to this demand of society. 


Itis in this frame of reference that evaluation is located and, specifically, the evaluation 
of the quality of University in Mexico. The foundations of the institutional evaluation complex 
were set in motion in the 1990s, and the purposes pursued with said complex are aimed 
at improving University by assuring society that the institutions and programs evaluated or 
accredited are of quality. 


The present work studies the theoretical foundations around the evaluation of 
the educational quality in the Public Universities of Mexico (UPM), using as a base the 
specialized literature in the reference topic. After a review of various perspectives of the 
evaluation, in the document an exercise of methodological synthesis is carried out on 
the evaluation process in Mexico, ffom which some critical elements of said scheme are 
established and, finally, an outline of a proposal is formulated. for the evaluation model 
of the PSUSs, in which it is suggested that evaluation must become an effective means for 
quality assurance and accountability. 


Thus, the development of this work is organized through five thematic sections, 
namely: |. Public University, Educational Quality and Evaluation: Theoretical Approaches; 
Il. Epistemological Research Framework; III. Evaluation Models in the Educational Context: 
Conceptualization and Characterization; IV. The Evaluation of the Quality of University in 
Mexico; and, V. Synthesis around the Evaluation of the Quality of Education in the Public 
Universities of Mexico: Criticism and Proposal. 


These five structures of the current document progressively intertwine to locate, first, 
evaluation and its link with educational quality in the context of University (Part |). Then the 
methodological perspective adopted in the essay is outlined (Part Il), and thirdly, evaluation 
in the educational environment is conceptualized, classified and modeled (Part III). 
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In the fourth section, a characterization of the evaluation of the quality of University 
in Mexico is established, considering the location of the Public University Institutions (IEPS) 
in the concert of the National Education System (Part IV). Subsequently, the fifth part of the 
work describes three moments (Part V): The first focuses on the presentation of a critique of 
the evaluation scheme of educational quality in the UPM. A second moment is oriented to the 
configuration of an academic evaluation proposal for the UPM, based on the consideration 
of a quality assurance and accountability approach; and, the third moment articulates, from 
a Synthesis exercise, the methodological and formal foundations of the evaluation of the 
educational quality in the UPM. 


Finally, a section with the Conclusions is presented, where the intention is not to 
reiterate the descriptive and critical aspects developed in the document, but, rather, to 
identify some elements that are considered relevant in the theoretical basis of this work. 


Introduction EEE 


PUBLIC UNIVERSITY, EDUCATIONAL QUALITY AND EVALUATION: 
THEORETICAL APPROACHES 


11 THE PUBLIC UNIVERSITY OF THE XXI CENTURY: CHALLENGES AND 
OPPORTUNITIES 

Currently, the university faces the challenge of adapting to the great changes that 
have taken place in recent decades in society. This task is a priority so that it continues 
to lead the training of youth and the creation of new knowledge, as it has been doing for 
centuries. 


In the 2i1st century, then, the university faces the following problems: The 
technological changes of the so-called information society, the growing demand for higher 
quality, the use of new training methods, the need to combine specialization with promotion 
of multidiscipline, university autonomy, etc. 


The dilemmas facing the university today have to do with how to make the social 
dimension of public service compatible with the quality demands that are typical of an 
institution dedicated to the development and transmission of science, technology and 
superior culture (Quintanilla, 1996). 


And it is in this new context that the university has the opportunity to demonstrate 
its ability to adapt and the social and intellectual leadership that it must maintain. For this 
reason, changes of great impact must be adopted both in the form and in the design of 
its internal organization and in its relations with society. And it is that, the main problems 
that this institution will have to face in the future come from these relationships, so the 
consequence of this will be the new relationship scheme between the university and the 
rest of society. 


An economic waste that no country can afford would be a direct consequence of 
maintaining a university with its back to social and technological changes. These relationships 
require the university to “teach to learn” in a new era in which today's knowledge becomes 
obsolete tomorrow and therefore openness, diversification, the need for reconversion and 
adaptability to the times must be features of a university education, at the present time. 


However, an important part of university activity is establishing a different link with 
society. This part tends to behave, in the long term, as an internal sector in the production 
system of a country, linked to the rest by a set of inputs (communication and information 
media, testing and research laboratories, etc.) and of outputs (applied research, training, 
particularly under the concept of “vocational training”, etc.) (Bricall, 1998). 

New problems arise daily in the life of universities. These are “the local demands 
for the creation of new universities, so frequent in recent years, or the decisions of the 
public powers to change, create, suppress and recombine university centers as if they 
were neighborhood schools, symptoms of the new situations “(Quintanilla, 1996). These 
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transformations are a product of developed countries and have important consequences 
such as the new conceptualization of the “University of masses”, a concept born to respond 
to the needs of a productive system and the right to education of all citizens in the actual 
society. 


And what characterizes the so-called mass university is that a large part of the 
population accesses university studies, with broader and more demanding social demands. 
Traditional universities were not designed for great social demand; there is a mismatch with 
the trends of current reality. 


Faced with this situation, universities, according to Rojo (1993), have to learn to 
face this growth without a deterioration of teaching or professional and research training. 
Autonomy and independence are essential when it comes to stimulating good teaching, 
training and research in public universities (UP). This autonomy is already linked to self- 
management and selíf-financing, to understand it at its best, but if it does not occur, then 
performance and results evaluation plans will be proposed, since investments must be 
linked to efficiency and social performance. This evaluation process will be dynamic and its 
purpose will be to improve the institution as a whole. 


21 UNIVERSITY IN THE CURRENT PUBLIC UNIVERSITY 


Today, the challenge of University is to appear as a system capable of responding to 
the demands of society, translating these into; increased demand for University, variation in 
the demand for professional qualifications, raising levels of quality and efficiency, etc. 


But perhaps the problem arises, as Guerci (1993) affirms: “when one thinks that it 
is possible to establish cuts in reality, resulting in: an education alien to the problems of 
its time, sterile when it comes to proposing alternatives of relevant life, because it cannot 
establish a dialogue between its theoretical constructions and the most basic needs of man 
in his daily life”. 

In the opinion of Rojo (1993), there are three reasons in modern society that must 
be satisfied by a University system: 1) Provide a high cultural level and scientific training 
to larger segments of society, which includes providing an element of professional training 
for those who currently do not progress in their educational trajectories; 2) provide effective 
training for the professionals who will make up the future backbone of society; 3) carry 
out research that contributes not only to advancing the frontiers of knowledge, but also to 
industrial development. 


Itis complex to combine these three tasks if you try to maintain quality at the same time 
and, on the other hand, it is doubtful that universities are the best choice when developing 
them, taking into account the considerations raised in the previous section. Consequently, 
according to Van Der Moler (1999), certain values associated with universities throughout 
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the centuries -such as those of independent judgement, creativity, and cultural and ethical 
dimensions- are probably of equal importance, if not greater, and that they must be based 
on scientific knowledge. Therefore, University must be understood both as a system from 
which to solve problems and develop attitudes, as preparation for learning, rather than 
accumulate knowledge for specific jobs. 


Thus, Van Der Moler (1999) proposes as essential activities in University: The 
creation, transmission and application of knowledge. And this proposal can be considered 
valid, but it must not be forgotten that the University systems of the different countries differ 
in so many things that there is no single solution that, in the near future, can produce a 
generally accepted uniform model. Most societies and governments value University as 
beneficial to society as a whole as well as to the individual. It is the success and great 
interest in University that has caused organizational and financial tensions within it, as well 
as with governments and students. The information available must allow us, however, to 


design feasible arrangements, given that they have political and social support. 


31 THE IMPORTANCE OF QUALITY FOR EDUCATION 


In recent times, the issues of Quality, Quality Assurance and Quality Management 
have become a priority concern and objective for organizations. 


Early on, it was the industrial world that first recognized that much of Japan's growth 
and economic success could be due to the insistence of Japanese companies on quality and 
quality improvement. It was especially from the eighties when the principles and methods of 
Total Quality Management began to be adopted in the European context to meet the level 
of quality desired by its customers and to continuously improve the quality of products. 
products and services they offer. 


In this context, it is not surprising that education has also shown its interest in this 
topic. Vocational training and continuous training, in the business environment, were the 
first sectors interested in learning about quality standards and quality principles. Both types 
of organizations are in fact closer to the needs of the market and to economic evolution than 
regulated educational institutions; however, this evolution is taking its first steps and both 
quality culture and quality tools are becoming unavoidable requirements in educational and/ 
or training contexts. 


Also, on the other hand, it must be said that this interest in quality is not new in 
the educational field: institutions, teachers, students, administrators and political decision 
makers have always been interested in this subject; that concern has extended to the use 
of standards and procedures that guarantee the quality of the teaching provided; although 
also traditionally, quality has been interpreted very narrowly, focusing only on particular 
characteristics of the educational or training services offered. 
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In this sense, the concept of quality has evolved over time. The current trend in 
quality is to focus more and more on the overall effectiveness of an educational provider, be 
ita professional school, a university or a private training institution. This trend runs parallel 
to the industrial evolution itself, in which quality considerations are shifting towards the 
organizational capacity to offer high-quality products and services. 


Broadly speaking, the new approaches to guaranteeing and managing educational 
quality imply (Van Der Berghe, 1998): a) A change in quality efforts in schools, which 
move from focusing on the quality of the teacher towards the study and optimization of 
the institution as a whole; b) the application of new or complementary quality control 
mechanisms in University; and c) The creation for the first time of quality assurance systems 
and performance-related mechanisms within continuing education. 

The current educational systems owe their configuration to the confluence of factors 
of various kinds: Social, economic, pedagogical, professional and cultural (Ruíz, 1998). 
All these factors have always existed but their relative importance has been changing 
over time in such a way that at present, the external demands of educational systems 
(ffom governments, students, businessmen, etc.) are increasing every time. more, putting 
pressure on the development of resources and the effectiveness of the organization. On the 
other hand, the continuous training sector is becoming a consolidated and mature economic 
sector within the productive services sector. 

These trends indicate that education is now more than ever beginning to be seen as 
a normal economic sector, which means that schools, universities and trainers in general 
must offer higher performance, behave in a more professional manner and constantly offer 
quality services. Itis shared with Van Der Berghe (1998: 8) that “education paradigms are 
changing from supply-driven teaching to demand-driven learning”. 

Among the specific factors that are contributing to this growing emphasis on quality 
in education, we can cite the following (González, 2000): 


a) Quality is a requirement of today's society. In other words, when it comes to 
quality, the wide range and high levels of quality of products and services presented 
by developed countries increase the demands of citizens and increase their critical 
spirit towards low-quality performance. Education has also participated in this spirit 
of demands. 


b) Quality is a factor of change, flexibility and customization. The society in which we 
live is versatile, people's ways of life and expectations are less uniform than a few 
years ago. In education this trend is seen through increasingly variable and complex 


qualification requirements. 


c) Quality leads to quality. The more knowledge is generated and information is 
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available on the quality of educational systems, the greater the demand for it. 


d) Quality implies commitment. Educational institutions are increasingly publicly 


responsible for what they do and for demonstrating that they offer a quality service. 


e) Quality as a means of survival. A well subsidized educational system will only 
be able to survive within a prosperous economy. In the current environment, any 
inefficiency or lack of flexibility will be penalized with fewer resources allocated to 


education. 


f) Quality involves many agents. In earlier historical contexts, the quality of education 
could be attributed almost exclusively to the inherent abilities of teachers. Due to 
the rapid evolution of the historical context, these individual capacities are no longer 


sufficient to guarantee a level of quality. 


9) The image of an educational institution is that of its quality. The prestige and 
value of an educational institution is no longer endorsed by its characteristics or 
historical reputation, but by the real quality of a particular institution. In this context, 
it is necessary to maintain a high-quality image for users and be able to permanently 


demonstrate its quality. 


h) Quality allows us to know the results of the training actions undertaken. To achieve 
a quality education it is necessary to invest in training and resources; In parallel, it is 


necessary to know the result of the training budgets in favor of quality. 


i) Quality requires transparency. The increasing variability and complexity of 
training proposals and plans makes it necessary to create mechanisms that enable 


transparency of the quality offered. 


j) Quality in its final state is projected in a culture of quality. The implementation of a 
quality system in an educational institution makes sense with the change of attitudes 
within the institution itself. 

Together, these factors call for greater attention to the quality of education and the 
development of mechanisms, procedures, and systems that can help ensure quality on an 
ongoing basis. Some of these factors are internal to educational structures, although most 
are external to them. Educational systems are increasingly in close interdependence with 
the rest of society and are therefore subject to corresponding pressures and trends. 


41 APPROACH TO THE CONCEPT OF EDUCATIONAL QUALITY 


Educational quality is a multidimensional concept and in the literature not all authors 
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use the term in the same sense. The task of doing so will be faced immediately, despite the 
difficulty that this entails, but it is believed in its convenience since, as Santos Guerra (1991) 
affirms, itis a term that is too often presented with a pretended univocity. 


From an etymological point of view, the concept of quality has two fundamental 
meanings: Understood as quality, it is identified as a set of attributes or properties referring 
to something or someone; secondly, it also refers to quality as superiority or excellence, as 
a degree that expresses the goodness of a thing. 


With De Miguel (1997) it is agreed that the concept of quality is relative and 
contextual since it can be defined from multiple perspectives, by different audiences with 
diverse interests and in very different contextual situations. Therefore, studying definitions 
that go beyond the generic level and that are located at a higher level of specialization, 
different approaches to the concept of quality will be presented in which certain elements 


are prioritized over others. 


From the thought of Bernillón and Cerutti (1989), quality lies in doing the job well 
from the beginning, responding to the needs of users, managing optimally, acting with 
consistency, a process or way of doing things; satisfy the customer/user; enjoy work and 
offer the best of oneself; reduce useless costs, avoid failures; and be more effective, efficient 
and productive. 


There are also other definitions that relate the characteristics of a product to the 
proposed objective, which combine the existence of certain innate attributes and user 
satisfaction, integrating all of this under the requirement of fitness for use. And according 
to this criterion, quality is defined as: “degree of coincidence with the requirements, such 
as the harmony between the properties of a product, material element and the purpose for 
which they are intended” (Salmerón, 1989). 


Most definitions refer to quality as satisfaction of needs. Such is the case of the 
concept given by the International Standard Organization (ISO) and the Spanish Association 
for Standardization and Certification, AENOR (1999), among others: “Set of properties and 
characteristics of a product or service that give it the ability to satisfy explicit or implicit 
needs. 


Also, López Rupérez (1994, 11), when referring to the term quality, does so in the 
following terms: “As if it were a magic word, the term quality evokes in people's minds the 
reference to a safe value, it is an attribute or set of attributes of the objects, services, or 
relationships that circulate within modern societies and that, according to the perception of 
the citizen, satisfy their reasonable expectations, making them trustworthy”. 


Inthe same sense, De Miguel and others (1994: 15), offer a classification in which they 
consider educational quality as a multidimensional concept, which can be operationalized 
based on very diverse variables. Thus, they present some of the options frequently used 
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according to Garvin (1984) and Harvey and Green (1993): 


1. Quality as an exception. 


a) Quality as something special distinguishes some institutions from others despite 
the fact that it is difficult to define it precisely. 


b) Classic vision: distinction, high class, exclusivity. 
c) Current vision: excellence. 


- Excellence in relation to standards. Reputation of the centers based on their 


means and resources. 


- Excellence based on scientific control over the products according to certain 


criteria: “centers that obtain good results”. 


2. Quality as improvement or merit. 


a) Quality as consistency of things well done, that is to say that they meet the required 


requirements: “Centres where things are done well. 


b) Centers that promote a culture of quality so that their results are better evaluated 


according to quality control criteria. 


3. Quality as fitness for purpose. 


a) Be part of a functional definition of quality, what is adequate or good for something 


or someone: 


- Centers where there is an adequacy between the results and the proposed 


aims or objectives. 
- Centers where programs and services respond to client needs. 


b) Centers that satisfactorily cover the objectives established in the legal framework. 


4. Quality as an economic product. 


a) Approach to the concept of quality ffom the perspective of the price involved in 


obtaining it: 
- Efficient centers by relating costs and results. 


- Accountability-oriented centers. 


5. Quality as transformation and change. 


a) Definition of quality focused on evaluation and improvement at the institutional 


level: 
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- Centers concerned with improving student performance and increasing 


added value. 


- Oriented centers the qualitative development of the organization was born 


(organizational development). 


And, in the same way, Rul (1995: 295, 318) also proposes his own classification, 
pointing out that the concept of quality is a myth of post-industrial societies and that it is 
used in education as a justification for certain policies. He also considers that any approach 
to the concept must represent a necessary demystification. In its conception, approaches 
to quality are approached too often in a reductionist manner, without going beyond formal 
statements and empty statements of effective content. It coincides that the concept of 
quality is clearly dynamic and transformational. 


In addition, according to the aforementioned author, the concept of quality applied 
to the educational field can be manifested in two main perspectives, namely: first, the 
axiological vision, referring to human potential; and the lifeworld perspective, linked to the 
ability to transform information into action. The first proposal derives from the Greek concept 
of “areté”, in the sense of excellence, perfection, value. Therefore, it means excellence and 
high-mindedness in the fulfillment or development of functions. It is an orientation modulated 
by the potential of experience and wisdom and is related to the Greek concept of “sophia”, 
in the sense of experience and insight. Thus, quality is the tendency to excellence based on 
the potential of people's experience and wisdom. 


The second option, on the other hand, refers to the management capacity of a person 
or a specific organization that transforms ideas and resources into beneficial realities. It is 
linked to the Greek term “techné”, understood as art or creation. In this sense, quality is 
the ingenious or artistic ability that knows how to influence a given reality, transforming it 
through the guidance of experience and knowledge. 


And, therefore, these considerations on the concept of quality ftom the perspective of 
different authors lead to the establishment of some basic characteristics of the concept: 1) 
Quality has a contextual value; 2) quality must be valued in a plural way; 3) the quality has a 
diachronic character; and, 4) the quality of education is not an exclusively technocratic task. 


Regarding the first feature, it can be said that quality, being related to temporal, 
ideological and economic parameters, makes it impossible to establish a single definition 
of the concept. The concept of educational quality, thus, is closely interdependent with the 
context in which it is used. The second characteristic points towards appreciating that quality 
is not a monolithic concept. It can be interpreted in relation to the achievement of objectives, 
level of user satisfaction, excellence in processes and results, proper development of skills 
and aptitudes, such as efficiency, self-realization, etc. 


Regarding the third feature attributed here to quality, according to the Organization 
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for Economic Cooperation and Development (OECD), it states that achieving quality can 
mean a radical and immediate departure from established provisions and practices and 
we set out at all times the objectives of the society and the institutionally valued purposes 
of education (OECD, 1991). And, from the fourth characteristic, quality is appreciated as a 
process imbued with values. Quality behavior does not occur automatically when there are 
some demands and knowing what things are valuable within the scope of those demands, 
which “joined valuable things become motives; furthermore, you have to choose which 
are the valuable ones and prioritize them” (OECD, 1991). This intrinsic characteristic of 
educational quality leads to the problem of political substantivity: for educational quality to 
be possible, it is politically necessary to believe that education is a social good. 


Lastly, in order to articulate a concept of quality, we will have to focus our attention 
on the justification of two fundamental criteria, namely: first, the concept of quality according 
to the prevalence of the educational process or product; and second, the concept of quality 
according to the educational field on which it is based. The following sections are dedicated 
to explaining these two criteria. 


4.1 Quality focused on the Process or the Product 


The context from which the concept of quality emanates has triggered that most of 
the definitions relate quality to results. According to Cobo (1985: 358), “an education will 
be of quality to the extent that all the elements that intervene in it are oriented to the best 
possible achievement.” 


For De la Orden (1993), educational quality is identified with a valid educational 
product; understanding validity as functionality, as congruence with the needs, expectations 
and educational aspirations of individuals and the community and that in practice is also 
effective. 


Another intrinsic difficulty results from the different meanings that the educational 
product can have (positive attitudes towards learning, acquiring and integrating knowledge, 
extending and refining knowledge, using significant knowledge, etc.). Definitions often focus 
on outcomes, equating outcomes with student performance. However, focusing only on the 
student dimension has the drawback of equating quality improvement with the academic 
results of schoolchildren. 


In this sense, Tiana affirms that “the evaluation, understood as a simple review of 
the final products, makes us lose sight of the consideration of the center as an ecosystem 
that allows us to explain and give meaning to the general functioning and the processes that 
develop the activity. In the same way, the consideration of one of the parts loses meaning 
isolated from the reference to the whole and from its connection with the context” (Tiana, 
1993: 290). 


From this conception it follows that what a movement towards quality must seek is 
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to improve the process that seeks results. From a perspective of quality centered on the 
process, Esteban and Montiel offer a definition of quality that offers a very approximate 
conception of the one we want to express in this work. Thus, they maintain that “Quality 
understood as a process is defined as a principle of action that does not exclusively aim 
at obtaining immediate final results, but, fundamentally, at a way of gradually doing things 
to achieve the best possible results in order to what is demanded of us and to the real 
possibilities and limitations that exist” (Esteban and Montiel, 1990: 75). 


The definitions of quality provided depending on whether they focus on the process 
or the product, suppose a limitation that has already been overcome in time. Currently, 
from a quality education of a multidimensional nature, the relationships between different 
elements of the context, the process and the product would be a common characteristic 
of the various quality educational manifestations. In this sense, we highlight the concept 
of coherence of Gil (1988) understood as interaction and sense of connection of all those 
elements that constitute the object of evaluation (educational program, school, educational 
system, etc.) and its indicators. Following this author, “the ultimate factor that determines 
the educational quality of a center is not the quality indicators considered individually, but 
the stability of the sense of connection that exists between them.” 


Therefore, the quality of education would be given by the coherence of each 
component with all the others and is characterized by the interrelation between three 
elements: effectiveness, efficiency and functionality. 


4.2 Quality according to the Educational Environment in which it is based 


The factors that have the greatest impact on educational quality according to an 
empirical study carried out by Cano García (1998) of a bibliometric nature, are: Teachers, 
Curriculum, Evaluation and School Organization. From this classification, we will analyze 
different definitions of quality related to these aspects. Thus, it is interesting to address 
quality focused on teachers, quality linked to training programs and quality oriented to 
institutional complexes. 


In the first place, the quality centered on the teaching staff has to do with the 
existence of affirmations that make the quality guarantee of the training of its teaching staff 
depend exclusively; From this perspective, the responsibility of the educational system is 
discharged on them and they consider that improving the quality of teaching staff requires: 
rigorous selection procedures that allow only the most qualified and highly motivated 
candidates to be selected; short, practice-based initial pedagogical training; a sufficiently 
motivating remuneration that prevents these professionals from fleeing to other professions; 
professional development plans and an administration that offers possibilities for promotion. 


Other elements that have been considered important are the stability of the teaching 
staff; teamwork and shared decision making; curricular planning and coordination as 


Public University, Educational Quality and Evaluation: Theoretical Approaches 13 


well as a high degree of autonomy. From this perspective, as Elliot (1993) points out, 
teacher evaluation can be interpreted as yet another strategy of coercive power. From a 
comprehensive model, educational quality centered on the teaching staff must serve both 
the improvement of the teacher and that of the institution. 


Secondly, quality focused on training programs. From this area of quality, the 
evaluation of the curriculum and of the teaching-learning processes will be fundamental 
objectives. Curriculum quality consists of planning, providing, and evaluating the optimal 
curriculum (according to the criteria of each country) for each student, in the context of 
a diversity of individual learners (Wilson, 1992:34). The quality of the teaching-learning 
processes is guaranteed when it enables, enhances and produces the result of more 
humanizing each and every one of the agents involved. 


And, thirdly, quality focused on centers and/or institutions. The quality of educational 
centers and institutions cannot be achieved through only political measures designed in a 
standardized and uniform manner. Our most recent history has shown that although these 
decisions have served as impetus and change, they have become incapable of equitably 
ensuring educational quality in the various schools in the country. OECD case studies 
demonstrate the extent to which the quality of teaching depends directly on teachers and 
schools, whatever the role of external measures taken by education authorities. De la Orden 
points out in this regard: “The quality of education, as soon as it is manifested in a valid 
product, will depend fundamentally on what happens in the school, on the structures and 
processes of educational institutions” (De la Orden, 1993: 264). 


4.3 Evaluation in the context of Education 


In the educational context, evaluation can be seen through its historical development 
and the changes that occur in different historical periods. Thus, in the following text, the 
evaluative process in education will have to be approached from making a generic historical 
synthesis around it. 


In a first period, which could be situated at the beginning of the 20th century, the 
evaluation is limited to the development of measuring instruments that make possible 
the evaluation of the student. The first evaluative practices in the educational context are 
therefore linked to the assessment of learning outcomes. In this period, evaluation is closely 
linked to the concept of performance measurement, and evaluation and measurement are 
therefore equivalent and interchangeable terms. At this stage, measurement (evaluation) 
is not linked to educational programs or the school curriculum, but constitutes a scientific 
projection of the scientific practices of the time, based on the positivist paradigm. 


In a second historical moment, located in the second quarter of the twentieth 
century, evaluation opens up to other educational dimensions. Tyler, in 1929, begins to 
consider evaluation with reference to objectives set out in curricular design. Although Tyler 
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primarily assessed learning, he made an important contribution by pointing out that the term 
assessment implied something more than the mere application of the test, and would serve 
to determine to what extent the objectives established in the curriculum had been achieved. 
This approach modifies, therefore, the initial idea] of measuring individual differences in 
terms of abilities and qualities of students and situates it in terms of curricular design. And 
this approach is considered at the beginning of the Program Evaluation. 


Continuing with the evolution of evaluation, around the 1950s, the idea of program 
evaluation resurfaced, motivated by the need to evaluate curricular projects in different 
areas that received strong state subsidies in the United States: in the 1960s, the Programs 
reach a social projection, expanding the concept of program evaluation beyond the strictly 
academic and/or school facet, originating a new profession in the United States. In Europe, it 
is Great Britain that in the sixties finances projects for the evaluation of curricular programs, 
under the idea that the evaluation must serve to help those who planned the curriculum. 
This type of evaluation adopts the name of Curriculum Evaluation, Curriculum Research or 
Evaluative Research. 


Going forward, the decade of the eighties represented a diversification, both from a 
structural and conceptual point of view. In the conceptual field, there was a shift towards 
more pluralistic positions, both in the methods or practices that are used, and in the models 
that guide them. One of the most specific and key changes is its political orientation. From 
the political point of view, the concept of evaluator has moved from that of “technical” or 
“expert” to that of a professional who performs an activity with a certain political impact. 
The evaluation goes from being considered a technical activity whose task is “to render 
accounts”, to adopting a political commitment or position. 


And, in the 1990s, political interest in evaluative activities was consolidated in very 
different fields and contexts: educational, social, political, institutional, etc. The evaluation, 
therefore, is recognized and warns of great interest at a scientific, political and social level. 


At the beginning of the 21st century, educational evaluation is characterized by 
the professionalization of evaluation and the diversification of evaluation practices (Arias, 
Verdugo and Rubio, 1995: 28). Thus, the professionalization of evaluation translates into 
the existence of training centers and activities aimed at training specialized professionals 
(universities, postgraduate studies, etc.), and the creation of professional associations 
and standards.' And, for its part, the diversification of evaluation practices is reflected in 
the diversity of professionals who assume these functions in terms of their fields of origin 
(education, economics, health, sociology, psychology), and the diversity of programs and 
methodologies that are apply. 


1 For example, the American Psychological Association (APA), which develops standards for its use in 
studies, or the Joint Committee Evaluation Standards, which periodically develops criteria for its applica- 
tion in evaluations, all together with the proliferation of specialized publications. 
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Structural diversity is manifested in the extension of evaluation to centers, institutions 
and educational systems, a fact that can be understood by the interest of educational 
policies, both at a local, national and international level, since evaluation is considered an 
important tool. to know the educational results and propose educational reforms (Glaser 
and Silver, 1994). 


Finally, one of the areas that has recently received great attention is the evaluation of 
educational systems. The identification and measurement of quality indicators of educational 
systems, centers and institutions constitutes a methodological challenge in this line. And, 
parallel to the extension of the territoriality of evaluation, questions and reflections continue 
to grow about the nature and forms of evaluation, as well as about the uses made of it 
(Moss, 1996; Schmidt, Jakwerth and McKnight 1998; and Wolf, 1998). 
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EPISTEMOLOGICAL RESEARCH FRAMEWORK 


11 GENERAL CONSIDERATIONS ON THE TERM PARADIGM IN SOCIAL 
SCIENCES 

Thomas Kuhn definesthe paradigm as “universally recognized scientificachievements 
that, for a certain time, provide models of problems and solutions to a scientific community” 
(Kuhn, T., 1975: 13). And Kuhn himself establishes that “a paradigm is what the members of 
a scientific community and only they share” (Kuhn, T., 1981: 318). 


In the same way, for Popkewitz, T. (1988), the idea of paradigm draws attention to the 
fact that science contemplates different conceptions, customs and traditions that constitute 
rules of the game that guide the research work. In this same order of ideas, Castillo and 
Gento define it as “the ideological reference framework or conceptual context that we use 
to interpret a reality” (Castillo and Gento, 1995: 27). And, Martínez conceives the notion of 
paradigm as the body of beliefs, assumptions, rules and procedures that define how to do 
science; they are the action models for the search for knowledge (Martínez, M., 1991). 


Therefore, the paradigm would be a structure constituted by a network of intertwined 
theoretical and methodological beliefs (patterns or operating rules) that allow the selection, 
evaluation and criticism of topics, problems and methods. Paradigms, in fact, become 
patterns, models or rules to be followed by researchers in a given field of action. 


And, the functions that a paradigm fulfills for Cook and Reichardt (1986: 61), are: in 
the first place, the paradigm serves as a guide for the professionals of a discipline because 
it indicates what are the problems and the important questions with which it faces. Likewise, 
the paradigm is oriented towards the development of an explanatory scheme (that is, models 
and theories) that can place these questions and these problems in a framework that will 
allow professionals to try to solve them. 


Third, the paradigm establishes the criteria for the use of appropriate tools (ie, 
methodologies, instruments, and types and forms of data collection) in solving these 
disciplinary puzzles. And, a fourth function of the paradigm is that it provides an epistemology 
in which the preceding tasks can be considered as organizing principles for the conduct of 
the normal work of the discipline. 


Based on the aforementioned, for the purposes of this work -which is developed in 
the context of the social sciences-, the paradigm is considered as a structure constituted 
by a conceptual theoretical framework and methodological beliefs shared by a scientific 
community. And in the social sciences, the concept of paradigm addressed here is valid, 
since it is observed that in the context of these sciences, there are different forms or 
theoretical and methodological tendencies to approach or explain a reality; therefore, the 
conflictive coexistence of different specific ways to do science is undeniable. 


Epistemological Research Framework 


21 RESEARCH METHODOLOGICAL APPROACHES 

Now, establishing a common thread with what was addressed in the previous section, 
the definition of three paradigms must be established immediately, based on the basic 
differences that comprise the ontological, epistemological and methodological aspects that 
frame the examination of the following questions: What is the nature of reality?; What is the 
nature of the relationship between the subject and reality? And how can the subject go to 
meet the knowable? (Guba, E., 1989). 


2.1 Positivism 

This paradigm, also called the quantitative, empirical-analytical or rationalist 
paradigm, has been considered the dominant model in the social sciences until recent 
times. Its purpose is to describe, explain, control and predict a reality. By its nature, the 
methodology used is experimental, manipulative and quantitative. 

According to this option, the facts and/or phenomena observed must deprive 
objectivity. The instruments to collect the information must be valid and reliable; the results 
are analyzed from statistical procedures that are used to make generalizations regardless 
of time and context to finally formulate theories among which are the following: 


1. Studies conducted within this perspective attach particular importance to careful 
research design, reliable measurement of variables, statistical manipulation of data, 


and careful examination of the evidence. 


2. Hypotheses are formulated to express the knowledge presumably obtained and 
these are considered confirmed when corroborated by deductive statistics that reach 


certain arbitrary levels of significance. 


3. The confirmed hypotheses (findings) can be generalized to other populations or 


contexts similar to the one studied. 


4. Simple relationships between the researcher's technical terms and concepts, 


research operations, research findings, and research conclusions are assumed. 


5. Social research is seen as offering the means of generating objective evidence 


and avoiding subjectivity and value judgements. 


6. Itis assumed that the results obtained are independent of the researcher in such 
a way that similar conclusions must be reached each time the same hypothesis is 
studied. 
In the school environment, the aspiration of the positivist paradigm is to discover 
the laws by which educational phenomena are governed, which lead to the formulation of 
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theories that guide action. In this regard, Carr and Kemmis (1988) point out that educational 
theories must be elaborated according to scientific criteria; for them, theory is what guides 
practice. Theories must be explanatory and guarantee that knowledge is objective, free 
from being contaminated by subjective preferences and personal inclinations and use the 
hypothetical-deductive method as a methodology. And for this, we start ffom a hypothesis 
that is formulated as general laws, and its validation results when comparing its deductive 
consequences with the results of observations and experiments. 


2.2 Interpretive Vision 


The interpretive vision, also called the qualitative, phenomenological, naturalistic, 
humanistic or ethnographic paradigm, encompasses a set of currents whose interest is 
focused on the study of the meanings of human actions and what happens in a given 
context. Eisner, E. (1998) points out that the purpose of (classical) research is to discover 
cause and effect relationships through experimentation or, if possible, through correlational 
studies that indicate correspondence between certain variables that are consistently related 
to others and that manipulation of one of them can induce changes in the others. While 
qualitative research tries to understand what teachers and children do in the groups in 
which they work, how they develop work based on observation, the use of the stories of the 
subjects themselves, to understand the experience of the subjects themselves research, 
description through the creation of written texts. 


Itis an attempt to bring out from within the subject the construction of meanings that 
allow feedback, the features related to the environment. The experience of the researcher 
is an aspect that must be paid attention to, the intelligent apprehension of the qualities 
of the context gives it meaning. The way to represent the findings is through the use of 
a prepositional language (discourse), which is built with assertions and which diminishes 
affective and personal traits. 


In this same sense, Arnal and others (1992: 41) speak that in the interpretive 
paradigm, “it is intended to replace the scientific notions of explanation, prediction and 
control of the positivist paradigm with the notions of understanding, meaning and action”. 
lts purpose is not to seek casual explanations of social and human life, but to deepen the 
knowledge and understanding of the reason for a reality. The purpose within this position 
is to reveal the meaning of things, through the systematic articulation of the structures of 
subjective meaning that indicate the ways of acting of individuals; that is, the objectivity of 
things is achieved by intersubjective agreements, which are defined, according to Florez, R. 
(1999: 10), as “certain features of the human being that allow him to understand himself and 
agree with others about the sense of words and actions that plan and coordinate between 
them”. 


Epistemological Research Framework [EEE 


2.3 Sociocriticism 


Regarding the sociocritical paradigm, Arnal et al. (1992: 41) summarize Foster (1982), 
that “critical theory was born as a response to positivist and interpretive traditions; With it, 
it is intended to overcome the instrumental and technical reductionism of the first and the 
conservatism of the second, admitting the possibility of a social science that is neither purely 
empirical nor only interpretive”; Their contributions, according to Pérez, G. (1998), originate 
“from community studies, female research and participatory research, among others.” 


This paradigm aims to analyze social transformations and respond to certain 
problems generated by them. Popkewitz (1988: 75) points out that some of its principles 
are: a) “Know and understand reality as praxis; b) unite theory and practice: knowledge, 
action and values; c) guide knowledge to emancipate and liberate man; [and,] d) involve the 
teacher through self-reflection”. 


Within the framework of this paradigm, the supposed neutrality of science is 
questioned, and therefore of research, which is attributed an emancipatory and transforming 
character of educational organizations and processes. In this regard, Pérez points out that: 
“just as education is not neutral, neither is research neutral; it is impossible to obtain impartial 
knowledge since the neutrality of science is false” (Pérez, G., 1998: 34). The socio-critical 
paradigm explicitly incorporates the ideological element; This orientation requires that the 
researcher be militant and thus incorporate processes of permanent self-reflection on the 
processes and situations investigated. Its objective is the analysis of social transformations 
and to respond to certain problems generated by them, which implies the generation of 
proposals for change, that is, to build a theory based on the reflections of praxis, as a critical 
analysis of doing. 


Finally, among some of the most important characteristics of the paradigm in 
question, Pérez synthesizes from Escudero (1987), the following: a) “It assumes a global 
and dialectical vision of the educational reality. Education is a phenomenon and a social 
practice that cannot be understood apart from the ideological, economic, political and 
historical conditions that make it up, and to whose development, to a certain extent, it 
contributes. b) Critical research assumes a democratic vision of knowledge as well as of the 
processes involved in its elaboration. In this sense, the research activity is a participatory 
activity in which both the researcher and the research subjects share responsibilities in 
decision-making. c) Underlies a particular vision of the theory of knowledge and its relations 
with reality and practice. Theory and reality are in constant dialectical tension. d) Critical 
research tries to articulate, generate and organize itself in practice and from practice. 
Research is built from a situational, social, educational and practical reality of subjects who 
have interests, concerns and problems. e) Critical research is decidedly committed, not only 
with the explanation of reality, nor with the understanding of the intelligibility that the subjects 
have of it, but with the transformation of that reality ffom a liberating and emancipating 
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dynamic of the individuals involved. in it” (Pérez, G., 1998: 34). 


31 PARADIGMS AND PERSPECTIVES OF EVALUATION 


The concept of paradigm, as well as its typologies addressed in the preceding 
sections, are closely related to evaluation for the purposes of this paper. And it is that, 
the discussion about the methodological procedures to carry out the evaluation process is 
linked to the paradigmatic positions that prevail in the research models. Authors such as 
Rodríguez, J and Beltrán, R. (1988), Fernández, M. (1995) Ruiz, J. (1996), Casanova, M. 
(1997) and Nevo, D. (1997), among others, classify the evaluation perspectives into two 
broad categories: quantitative and qualitative. 


First of all, the quantitative perspective is linked to the quantitative trend of evaluation 
applied to the educational field, and is based on the application of procedures that require 
the measurement and quantification of educational processes and/or products. Through 
this type of evaluation, appropriate procedures are used to guarantee the greatest possible 
objectivity in the information. And, in general, their results are expressed numerically and 
there is the possibility of establishing generalizations from the data obtained. 


On the other hand, the qualitative perspective is articulated with the qualitative trend 
of evaluation applied to the field of education. Those who advocate qualitative procedures 
consider each educational phenomenon as something unique, which is conditioned by the 
context in which it occurs, making it difficult to generalize its results. Casanova points out 
that: “this situation of inadequacy occurs when it is necessary to evaluate processes, which 
occurs in most educational situations to a greater or lesser extent” (Casanova, M., 1997: 
111). 


Finally, with this type of evaluation, educational phenomena are studied in their 
natural context and procedures are used to capture them comprehensively (Cook and 
Reichardt, 1986; and, Rodríguez, J. and Beltrán, R., 1988). 
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EVALUATION MODELS IN THE EDUCATIONAL CONTEXT: 
CONCEPTUALIZATION AND CHARACTERIZATION 


11 CONCEPT AND CHARACTERISTICS OF THE EVALUATION PROCESS 


In the context of education, speaking of evaluation means speaking of goals, 
objectives and criteria, of theories and models, of all the components that participate and 
compete in the educational system (Mufioz, et. al., 1995). Throughout our modern history, 
the concept of evaluation has undergone progressive changes, which is why there are also 


numerous definitions of evaluation that exist. 


Some will focus on determining to what extent educational objectives are achieved 
(Tyler's studies -1950- are a clear example of this); others focus on the conception that 
evaluating is providing information for decision making (in this line we have the studies by 
Cronbach -1963-, Stufflebeam -1969- and Alkin, 1969); evaluation has also been considered 
as the assessment of merit or value (Scriven -1967- and Eisner, 1979), Or already in the 
eighties, we have that evaluation is understood as an activity that includes, on the one hand, 
the description understood as a collection of data and, on the other, the critical judgment, that 
is, it is the values, social norms and preferences of the individuals who finance the evaluation 
(Guba and Lincoln, 1982). And in the nineties, evaluation is understood as improvement (De 
Miguel, 1994; Mateo, 1995; Pérez Juste, 1995; Álvarez, 1988; and López Mojarro, 1999). 


In 1986 and 1997, Nevo points out a series of issues that must be taken into account 
when conceptualizing evaluation, namely: 1) the definition,! 2) the objects,? 3) the type of 
information,* 4) the criteria, 5) the functions, 6) the hearings,* 7) the evaluation process,” 8) 


1 Regarding the definition, for Nevo (1986 and 1997), evaluation must be understood as the systematic collection of 
information regarding the nature and quality of educational objects. 

2 The educational objects to be evaluated will be all those definable entities, in this sense we will be able to evaluate the 
educational programs, the curriculum, the educational centers in general, the staff and the students. 

3 And regarding the type of information, it is not only interesting to obtain information on the results of the evaluation, or 
the educational objectives, or those aspects more related to decision-making, as we saw in the previous definitions, but 
itis also necessary to examine and obtain information needs, objectives, strategies, implementation processes and the 
sociopolitical environment. 

4 Regarding the criteria, Nevo refers to five criteria for judging merit or worth: internal consistency, comparison with 
alternative objects, previously agreed standards, acceptable social norms and values, and critical expert judgments. In 
addition, if we emphasize this issue, De Miguel (1994) understands that the evaluation criteria must be linked to the prin- 
ciples that guide the activity of the center, the specific objectives that are intended to be achieved with these principles 
and the evaluation criteria. For this author, the evaluation criterion is a quality objective with respect to which a pheno- 
menon or a specific aspect of the educational center can be assessed. And, in this line, López Mojarro (1999) points out 
that a teaching center has clearly defined intentions by current legislation, by its own projects and by the ideas that give 
it character. It also has some established objectives, these being the ultimate justification for the existence of the center. 
For Pérez Juste (1997) the criteria to which Nevo refers are simple references when assessing. Consider the criterion 
as the norm to judge the value of something or, in other words, that aspect that from that something is valued as correct 
or adequate. And, in summary, the concepts of principles of the center, objectives and criteria are outlined as necessary 
elements when defining the evaluation. 

5 Regarding the functions, it can be noted that there are numerous functions that the evaluation can fulfill. The most clas- 
sic conceptions range from evaluation focused on improvement (formative) to one focused on selection, certification or 
accountability (summative). Other conceptions are aimed at increasing motivation (this function is more of a sociopolitical 
nature since evaluation is used to motivate and gain public support) or to exercise authority (an administrative function). 
In any case, as Nevo points out, the basic function of evaluation will be to help understand the nature of the object of 
evaluation and its quality or, as López Mojarro (1999) refers, the function of evaluation is improvement. 

6 Regarding the audiences, these can be considered as all the possible groups related to the object to be evaluated, 
as well as all those who for one reason or another have an interest in the results. And it is necessary that the evaluator 
meets the needs of the interested parties. 

7 On the other hand, regarding the evaluation process, different procedures could be suggested depending on the 
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research methods,º 9) types of evaluationsº and 10) evaluation standards.'º And this series 
of questions lead to consider the concept of evaluation from a global point of view, that is, 
that it is applied to the entire educational process and that takes into account something 
more than the teaching-learning processes, that is, the educational center. In its whole. 


In other words, evaluation is not a sum of paris but rather, as Scriven (1991) writes, a 
new discipline with its own body of knowledge. It is not only an essential tool to help detect 
what is happening in an educational center, but it puts us in a situation of what it needs to 


improve. 


Thus, to evaluate is to know with the intention of improving, it is to know the facts 
and the factors that condition them, it is to know systematically (López Mojarro, 1999). And 
this systematicity is what makes it not only an instrument but also, as Ruíz (1998: 18) points 
out, a “process of structured and reflective analysis, which allows understanding the nature 
of the object of study and making value judgments about it. the same, providing information 
to help improve and adjust the educational action”. 


Within this line, De Miguel (1994) goes a little further and considers that the evaluation 
is not relegated to the end of the process but is inherent to the process, which makes 
assessment an integral part of learning, teaching and school management. The results 
of the evaluation become the elements that help define and redefine the objectives and 
encourage teachers and students to take responsibility for learning, teaching and managing 
the centers where they study and work. 


Therefore, the evaluation understood as a process, according to Rotger (1989), 
brings together a set of characteristic features, namely: 1) the evaluation must be perfectly 
integrated into the curriculum vitae, so that this is one more phase of the process; 2) it must 
have a formative component, and this character will be imprinted both in the educational 
process and in the results; 3) it must be continuous, which means that the effects of the 
evaluation must be known throughout the entire process, not at the end, in order to make 
the pertinent corrections to achieve the greatest possible number of objectives set at the 


approach from which we start. And following the definition of Nevo noted above, we can point to the following stages or 
phases: a) understand the problem under evaluation, b) plan the evaluation, c) collect data, d) analyze data, e) report 
on the results and f) provide recommendations; understanding this process as cyclical where each phase is subject to 
being repeated throughout the evaluative study. 

8 As for research methods, everyone knows the problem between quantitative and qualitative methodology. What must 
concern us is not whether we must use one or the other, but rather which methodology responds to the problem | raised 
and which responds better to my expectations of success. This will force us to apply the principle of methodological 
complementarity in order to be able to contrast the data with varied and complementary techniques. 

9 On the other hand, regardless of the type of evaluation from which we start, be it internal (self-assessment, hetero-as- 
sessment or co-assessment) or external; initial, procedural or final; global or partial; summative or formative, evaluators 
need to be trained in the following skills: a) understand the social, organizational and personal context of the evaluation 
as well as the specific characteristics of the object of evaluation; b) have communication skills that allow them to es- 
tablish empathetic relationships with the human groups they work with; c) have knowledge of evaluation methods and 
techniques. 

10 And, around the evaluation standards, the Joint Committee on Evaluation Standards in Education developed a set 
of standards divided into four groups: utility standards (so that the evaluation meets the practical needs of information); 
feasibility standards (they will ensure that the evaluation is real and prudent); property standards (they will ensure that 
the evaluation complies with legal precepts and is developed ethically) and, finally, precision standards (they will ensure 
that the evaluation is technically adequate). 
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beginning; 4) it must be recurrent, that is, through the feedback dimension it provides, the 
process will be improved based on the results obtained (it is closely related to the previous 
characteristic); 5) it has to be criterial, in relation to some objectives set at the beginning 
and that lead the process towards them; 6) it must be decision-making, which means that 
decision-making must allow the improvement of the learning process and results; 6) it must 
be cooperative, insofar as the evaluation concerns all the levels involved in it, they must 
participate in its process. 


To the above characteristics, Cardona (1994) and Ruiz (1998) add its indirect nature 
(given that the variables in the educational field can only be measured as long as they 
are observed), scientific (in its instruments, methodology and data analysis) and referential 
(because its purpose is to refer to the achievements obtained with the proposed objectives). 
And as a direct consequence of this vision of understanding evaluation, it is feasible to 
significantly sensitize all the agents involved in education, understanding that education 
is a public good that it is important to improve and care for; With which the administration 
and public powers have carried out a new ordering of the educational system, increasing 
public investments in the educational field. And the foregoing has motivated the concept of 
educational evaluation to encompass students, teachers, centers and, in conclusion, the 
educational system as a whole (Hernández Pina, 1995). 


21 CLASSIFICATION OF EVALUATION MODELS 


Like the epistemological approaches, the conceptions of educational evaluation 
respond to a historical moment and of course are related, but in practice this relationship 
is often ignored. Some of these conceptions will be characterized by being highly complex, 
however, they are very useful, since they lead to obtaining results with a wide margin of 
validity and reliability; others, on the other hand, are not so complex to put into practice, but 
their results are impregnated with a high load of subjectivity. Finally, other approaches that 
reflect a mixture of methodologies without considering the purpose that ultimately frames 
the research can be noted. 


When dealing with the evaluation of educational programs, Castillo and Gento (1995: 
27) point out that the evaluative paradigm “supposes a general conceptualization of the 
most appropriate way to carry out the evaluation. The most appropriate evaluative models, 
methods, techniques, procedures and instruments are derived from this conceptualization”. 
And likewise, there are various classifications to organize and describe the different 
approaches or evaluation models. In the case of the work that is being addressed, the term 
model will be used in order not to get lost in this labyrinth of methods, approaches and 
procedures. This limitation is made in order to clarify the confusion that exists about how to 
approach the conceptualization of evaluation and determine its profile in education. 


Next, four typologies of classification of evaluation models in the educational context 
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will be addressed, namely: Poham's classification (1980), Pérez's (1989), Castillo and 
Gento's (1995) and Monedero's (1998). 


2.1 Poham's Classification 


There are four categories proposed by Poham (1980). He establishes four 
descriptive categories: 1) The Goal Achievement Model, which conceives evaluation 
basically as the determination of the degree to which the goals of a teaching program are 
achieved; 2) Judgment Models that emphasize intrinsic criteria, in which the evaluator 
exerts considerable influence over the nature of the evaluation, since it is the evaluator's 
judgment that determines the favorable or unfavorable evaluation; 3) Judgment Models that 
emphasize extrinsic criteria, where the latter are associated with the effects of the object; 
and the Models that facilitate decision making, whose main task is to collect and present 
information that allows the analysis of decision alternatives. 


2.2 The Perez Classification 


Pérez (1989) analyzes the assessment approaches and models that have dominated 
teaching theory and practice into two large groups: quantitative experimental models and 
alternative-qualitative models. 


The former have dominated evaluative research and practice for most of the 
20th century. They are characterized by the search for objectivity, by the applicability 
of the hypothetical-deductive method, by the rigor of the statistical methodology, by the 
almost exclusive emphasis on products and by the quantification of information through 
objective means and instruments, without identifying the individual differences in treatment 
of outcomes. Among these models are: the systems analysis approach, evaluation as 
achievement of objectives, evaluation as information for decision making and evaluation 


without reference to objectives. 


Regarding the alternative-qualitative models, they present an approach that seeks 
options to understand and assess the relevance and meanings of a teaching-learning process 
that goes beyond the observable results, previously established as an object of evaluation. 
and is characterized by maintaining a criterion relative to objectivity, since the evaluation of 
a phenomenon or set of events is subject to limitations and errors, by making descriptions 
and interpretations with a global, understandable and natural sense and by the use of the 
holistic method- inductive that gives the evaluation a participatory character. These include: 
evaluation based on the artistic criticism model, models based on negotiation, illuminative 
evaluation, case studies, stakeholder evaluation and democratic evaluation. 


2.3 The Castle and Gento Classification 


For their part, Castillo and Gento (1995) approach a classification that admits three 
models: the behaviorist-efficiencyist, the humanistic and the holistic. 
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In the first place, the Behavioral-Efficientist Model focuses on the evaluation of the 
objectives taking into account the profitability of the resources based on productivity and 
efficiency. Examples of this model are: Tyler's goal attainment; the C.I.P.P. of Stufflebeam; 
the C.S.G. from Alkin; and Cronbach's educational planning. 


Second, the Humanistic Model focuses on valuing individuals and the activities 
they themselves carry out. They have also been called subjectivist or phenomenological 
models.' 


Finally, the Holistic Model, in which the evaluation focuses on comprehensively 
and globally assessing the various components that make up a program. Example: Stake 
Respondent Studies, Parlett and Hamilton's Illuminative Assessment, and McDonald's 
Holistic Assessment. 


2.4 The Wallet Classification 


Monedero (1998) groups the evaluation models into two large categories, namely: 
the classic models and the alternative models. 


The Classical Models are those models that are framed in the empirical-rationalist 
and behavioral orbit. They handle numerical data and analyze it, frequently, through 
statistical procedures. Among them are: 1) Models based on the achievement of goals or 
objectives; 2) Models based on the formulation of judgments; and 3) Facilitating models for 
decision-making. 

For their part, the Alternative Models are made up of all those models that have 
appeared, generally, as a reaction to those previously mentioned. Almost none handle 
numerical data, they use another type of analysis and focus on the evaluated reality. 
Examples of this typology are the following: 1) evaluation as artistic criticism; 2) goal- 
free evaluation; 3) modus operandi method; 4) communicative evaluation; 5) illuminative 


evaluation.; and corresponding evaluation. 


31 EVALUATION MODELS IN EDUCATIONAL INSTITUTIONS 


In order to approach this section, it has been decided to consider the assessment 
models from three different perspectives. It is intended to obtain, on the one hand, a temporal 
vision of the various models from the perspective of the organizations. Subsequently, they 
will be analyzed from an epistemological and methodological point of view. And, finally, a 
review of the current evaluation models will be made based on the quality criterion. 


3.1 Organization Evaluation Models 


Currently, it is assumed that educational institutions or centers must be taken 


11 For example, the Owens studies; Eisner's art criticism; and Scriven's customer service model. 
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as organizations to the extent that they have a complex structure and that it cannot be 
analyzed from partial approaches. Current research focuses on analyzing one's own culture, 
personality or functioning from different methodological points of view. 


All the theoretical schemes around organizations refer, in one way or another, to 
an evaluation process that allows, on the one hand, governments to know if educational 
institutions, school organizations, fulfill their missions or not, to the rather than whether 
the resources allocated to them are used effectively and efficiently. On the other hand, 
the evaluation allows the institutions themselves to verify whether the proposed objectives 
have been achieved and to detect the internal dysfunctions that prevent obtaining more 
satisfactory results. That is to say, the efficacy is verified and those elements of dysfunction 
and susceptible to change are identified (evaluation for change). 


When talking about evaluation models referring to organizations, what is being 
postulated is, in the words of De Miguel (1989: 30) and Mateo (1995: 75), a theory about 
its structure and functioning that is specified through variables related to each other and in 
connection with others that we will establish as indicators of efficacy and quality. Determining 
the criteria of quality or effectiveness that organizations must assume constitutes the 
central point of any evaluation, so we must refer to the theoretical approach from which 
it is conceptualized. In the words of these authors, the relationship between theory and 
evaluation is so close that we can understand the existence of as many evaluation models 


as theoretical approaches are formulated. 


This leads us to think that every evaluation of organizations has a model. It must not 
be forgotten that the model is a guide, it is an ideal and abstract representation of a reality 
based on theoretical references. As a theoretical-hypothetical representation of a reality, 
it must have certain characteristics: a) it must be based on a theory; b) the variables with 
which it works must be capable of being specified operationally; c) the model must have 
empirical validity and d) the applications of the model must be validated and generalized. 


To make a classification of these we could take various criteria. Firstly, we will choose 
to review the classification presented by De Miguel (1989),'2 since his vision will allow us 
to analyze what was the past and what is the future of research in educational institutions. 
Subsequently, an outline will be made of the main existing evaluation models based on 
another criterion. 


3.1.1 Results-focused models 


12 Although the scheme to be followed is the one presented by De Miguel in a work published in the “Revista de Investi- 
gación Educativa” in 1989, this author makes another more reductionist classification than the one exposed in his 1994 
work: “Assessment for the quality of secondary schools” and published by Escuela Espafiola, in which he summarizes 
the models in two large blocks: models centered on efficiency (in which, following Scott -1981- Shortell and Kaluzny 
-1983- he analyzes institutions based on the structure, processes and behaviors from three approaches: rational, natural 
and open) and models focused on the evaluation of change. 
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The purpose of these models is to assess mainly the effectiveness of institutions 
and organizations but from the perspective of results (external perspective) expressed 
in quantitative terms, using input-output designs and correlational analysis techniques. 
Most of the evaluative works focused on effectiveness fall within this line. Estimating the 
performance of an organization leads to evaluating the results based on the proposed goals 
(effectiveness). 


One of the advantages of this type of model is that it provides data on causes and 
effects, identifying the characteristics of the centers but not discriminating which variables 
influenced efficacy. And, among the disadvantages that could be pointed out in these 
models is that they do not allow evaluating the value of the goals, impose difficult controls 
for the educational context and have serious conceptual and methodological limitations 
(Fernández, 1990). 


Among the works that can be cited within this line are the following (Mufioz, 2000: 
167): de Averch (1972), Brookover (1974), Rutter (1979 and 1983), Madaus (1979), Purkey 
and Smith (1983), and Lotto and Cunning (1984). And, based on these references, the 
criteria or indicators of effectiveness success normally used are: 1) evaluation based on 
the objectives, where effective schools are those that meet the planned objectives; 2) 
evaluation in relation to resources, where the evaluation criterion of effective schools lies in 
the ability to acquire, generate and use resources in order to produce more and better; and 
3) indicators of a social nature, where the starting criteria is the contribution and/or services 
that organizations provide to society.'º 


3.1.2 Models focused on the Efficiency of Internal Processes 


The models centered on this point of view consider organizational evaluation as an 
efficiency process from a double perspective. On the one hand, efficiency in the internal 
functioning -structure and operations of the organization-. On the other, the satisfaction 
of the members of the organization. Assessing the performance of an organization from 
efficiency means estimating these results based on their costs. 


They are models focused on the internal efficiency of the school (note that there is 
a shift from previous models whose objective was efficiency), which implies an interest in 
internal aspects of the organization itself. There is an interest in analyzing psychosocial 
factors such as the organizational structure, the director, the climate and the classroom 
processes. Its objective is to analyze the operation and satisfaction of the members of 
the center. They respond to an explanatory methodology with input-context-output designs 
(using regression analysis) and comparative designs. 


Among the advantages it presents is that they take into consideration all the 


13 The indicators will depend on the political guidelines on the function of the school at each moment. 
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variables. And, on the contrary, they continue to have certain conceptual and methodological 


limitations. 


Among the authors that could be pointed out within this line, there are: Goodman 
and Pennings (1977), Van De Ven and Ferry (1980), Hall (1987), Schoemoker and Fraser 
(1987). And, these models take three indicators as a reference: 1) the role of the director 
and the team or staff; 2) institutional efficiency; and 3) the institutional climate. 


Studies on the role of the director and the team or staff point out that any evaluation 
of organizations cannot do without the practical aspects related to the management and 
administration of the organization. These models see the director as the person responsible 
for the effectiveness of the organization insofar as it influences the achievement or not of 
goals, school climate and institutional efficiency. 


For their part, the models on institutional efficiency use as indicators those variables 
implicit in the dynamics of the classroom such as the opportunities that the student has to 
learn, the distribution of time, the teacher-student interaction, the proposed curriculum. The 
models focused on the institutional climate are based on the fact that the internal processes 
that occur in an organization have as a consequence a relationship between its members 
that has an implication with their degree of personal satisfaction and, therefore, the 
effectiveness of the organization. And, they are indicators of these models: the processes 
of internal integration and cohesion, the perceptions, attitudes and expectations of teachers 
and students, forms of communication, among others. 


3.1.3  Causal Models 


If the previous models are analyzed, some try to explain organizations from an 
external perspective (analysis of results) and others from an internal perspective (how the 
organization works). However, it is extremely difficult to dispense with one of the process 
variables and the other with differences. 


To overcome these limitations, causal models are bom as an integrating response 
that allows us to analyze the cause and effect relationships between the variables related 
to the process and those related to the results. What these models intend to study is to 
determine the pattern of causation of the efficacy of the center, through causal analysis, 
analysis of covariance structures and confirmatory-exploratory factor analysis. 


These types of studies have the advantage of allowing cause-effect relationships 
to be verified, although there is a clear difficulty in evaluating the processes. And, some of 
the authors that can be referred to along this line are: Yorke (1987), Stõel and Scheerens 
(1987), De Miguel (1994) and Fuentes (1988). 


3.1.4 Cultural Models 
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The estimation of schools as social systems requires considering the cultural 
aspects that are implicit in an organization. Thus, cultural models start ffom ecological, 
phenomenological, ethnographic, etc. perspectives, focusing their analysis on aspects such 
as communication, thoughts, interactions, images about the center, within what is known as 
an emerging paradigm. 


Therefore, this typology of models can be classified into three sections, namely: First, 
culture is estimated as a set of information elements about individuals and groups within 
a population; second, culture is considered as conceptual and symbolic structures from 
which its members build the reality of the school; and, third, culture is conceived as a set of 
purposes and differential meanings generated within a specific group in its relationship with 
the social and political environment. 


3.1.5 Models focused on the Evaluation of Change 


Parallel to conducting research on effective organizations, other work was carried 
out with the goal of innovation and change. These models sought to improve the school 
based on the fact that all those involved in their activity can introduce changes in the pro- 
cesses and procedures in order to improve its organization and internal functioning. Thus, 
such models are considered as the alternative towards which the evaluation processes of 
educational centers must tend, with the objective of finding out those aspects of the centers 
that must be improved and how they can do so. 


For these reasons, the models in question have a control function (efficacy or 
achievement of objectives is controlled), diagnosis (of those elements that produce 
dysfunction) and intervention (in order to achieve improvement). And, in this context, 
from organizational theory, success is given by the number and quality of the products or 
objectives achieved and, on the other hand, by the processes of improving the effectiveness 
and quality of life within the organization. 


The reference models have involved changes at a conceptual and methodological 
level, and among those that De Miguel (1994) points out, are the following: a) From a 
conceptual point of view, there is a shift ffom causal models to others of a contextual nature. 
This means that from designs that seek the cause-effect relationship, we move on to change- 
oriented approaches whose purpose is to analyze the relationships of phenomena in an 
interactive and procedural way; b) if the previous objective was to analyze the performance 
of the students, now the aim is to diagnose the internal problems of the institution in order 
to proceed to solve them. The process will focus on a series of indicators that allow the 
institution to be observed; and c) the models for change consider each institution as unique, 
with its own autonomy. Which gives you the right to start your own improvement processes. 
That is, part of the self-reflection or self-assessment. 
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The current theory on the evaluation of the institutions tends towards an external- 
social evaluation to verify its effectiveness and, towards an internal evaluation to identify 
those dysfunctional aspects of the centers. These are designs aimed at improving schools 
based on procedural approaches where self-assessment is the backbone and in which the 
evaluator does not have to be an external element to the institution. This is the approach 
proposed by the OECD (1991) and updated by the CERI (1994) with a view to achieving 
effective schools. 


3.1.6 Evaluation Models from Epistemological Perspectives and 
Methodological 


In correspondence with the different ways of conceiving the success of the centers, 
different models or paradigms of evaluation have originated. In reality, however, it is very 
difficult to find pure models that can be located in a certain conception since they all have 
their own way of understanding what evaluation is, what its scope and purpose are, and how 
it can be carried out. 


Thus, from a methodological point of view, there has been a tendency to point out the 
existence of two groups of models: on the one hand, there are the classic models oriented 
towards the achievement of goals (as in the case of the Tylerian proposals) or those oriented 
towards the decision making (Stufflbeam's CIPP model); and, on the other hand, there are 
alternative models, such as the alternative model of Stake, Hamilton's Illumination, Eisner's 
artistic criticism and McDonald's model of democratic evaluation.'4 


However, in the following spaces, these models will have to be explored in greater 
depth, starting from the classification articulated by Colás and Rebollo (1993) and Medina 
and Villar (1995), based on the development of Stufflebeam and Shinkfield (1987). This 
classification roughly distinguishes three moments: efficiencyist or objectivist models, 
humanistic, phenomenological or subjectivist models, and holistic models.'º 


3.1.7 Efficient-Behavioral Models 


In this type of models could be framed those that De Miguel (1989) pointed out as 
those centered on results, in models centered on efficiency and causal models. His concern 
was focused on the evaluation of the objectives and the profitability of the results. And it is 


14 Popham (1980) classified appraisal models into goal-attainment models, judgmental models emphasizing intrinsic 
criteria, judgmental models emphasizing extrinsic criteria, and decision facilitation models. De la Orden (1993, 1995b) 
distinguishes two modalities or conceptions of school models depending on whether or not they refer to the product. 
Those that take into account the product are structured according to a series of variables selected based on their rela- 
tionship with student performance. Those that do not refer to the product are not considered as diagrammable models. 
Thus, among the former, he distinguishes: input-output models, process models of institutional efficacy, process models 
with an emphasis on class, and systemic models. 

15 Likewise, Colás and Rebollo (1993) carry out a very similar classification: subjectivist models, objectivist models and 
critical model. 
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that, what this type of models intend is to control the variables (with evaluative designs of 
experimental research, of a scientific-technological type) that affect the results to achieve 
the best product, making the most of resources. 


In addition, the characterization of efficient-behavioral models is given by the 
following distinctive features: 1) The evaluator, who is generally external and acts as a 
specialized technician; 2) the techniques or instruments used are closed questionnaires, 
standardized tests, objective tests (quantitative methods), in which objectivity is sought; and 
3) the weaknesses that they suppose have to do with the fact that human interactions are not 
contemplated, since the educational centers do not allow a total and rigorous experimental 
control. Among the options of these types of models, the evaluation model based on Tyler's 
objectives (1967), the eight-stage evaluative procedure of Metfseel and Michael (1967), the 
educational planning model of Cronbach (1980), the model Context-Input-Output-Output 
(CIPP) of Stufflebeam and Skinkfield (1987) and the figure model for educational evaluation 
of Stake (1975). 


In his evaluation model based on objectives, Tyler (1967) considers that the 
evaluation must determine the congruence between work and objectives. For this author, 
the evaluation process is one that determines to what extent the educational objectives 
were achieved through the curriculum and teaching programs. In addition, Tyler sought to 
establish, classify, and define goals in terms of performance. 


The reference model, likewise, is oriented towards decision-making, which must 
be in accordance with the objectives and the results obtained, using assessment tests to 
measure the achievement of the planned objectives, the personal rate of evolution of the 
individuals, and the adaptation to parameters or norms established in the sociopolitical field 
in question (Medina and Villar, 1995: 47). And, despite the fact that this model has been 
criticized above all because the choice of objectives is restricted to performance objectives, 
with which the evaluation tended to become somewhat incomplete, it is worth noting the 
enormous importance that this author gives to evaluation to judge the process of a program 
as its final results, laying the foundations for the first systematic method of evaluation 
(Stufflebeam and Shinkfield, 1987). 


On the other hand, Metfseel and Michael (1967) developed an eight-stage evaluative 
procedure aimed at helping school personnel to evaluate the achievement of objectives, and 
through which these authors agreed with Tyler that the evaluation of the results could make 
educational decisions more perfect. The eight stages of this evaluation paradigm are based 
on: a) The involvement of all members of the school community in the evaluation; b) make 
a prioritized list of goals and objectives; c) translate the specific performance objectives 
in a way that they can be put into practice to facilitate learning; d) select the instruments 
that allow verifying the effectiveness of the educational processes in order to analyze if the 
objectives are met; e) apply the instruments to analyze if there are changes in performance 
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with respect to the selected objectives; f) analyze the data through statistical methods; 9) 
interpretation of the data; and, h) make recommendations to the corresponding audiences. 


Thus, the authors of reference make professional evaluators, students and teachers 
responsible for the evaluation in each of the stages and to a different degree. And, they also 
insisted that those decisions that involved any prosecution must be included in all stages 
of the evaluation process so that those involved in each phase make their readjustments 
based on the information received. 


In addition, Cronbach (1980) outlined a model of educational planning, a proposal that 
arises from this author's interest in the evaluation of social science programs, considering 
that evaluations are designed to fulfill a political function, for which there was that stimulate 
their planning. 

According to Cronach, the evaluator must not be unique, so the responsibility for 
the evaluation must be held by a team that, on the one hand, distributes priorities and 
responsibilities among them and, on the other, details its internal planning. Although the 
evaluation is usually carried out at the request of the administrators, for an external evaluator 
to apply sampling, assessment and analysis techniques, the evaluators must strive to reach 
the citizens in such a way that it clarifies the problems of those involved and provides 
information (clear, timely, accurate, valid, and comprehensive) and troubleshooting tips. To 
achieve this, the evaluator must have an open mind as well as advantages in communication 
and must attend to the units (any individual or class), to the treatments (which affect the 
units) and to the observation operations (data collection techniques, data analysis, etc). 


And it is that, according to this author, the evaluation must deal with something more 
than the objectives, it must be considered as a study of both process and results. It also 
emphasizes how the results obtained in a given field can be used for decision making in 
other fields. 


If Tyler's model was developed in the early 1940s and Cronbach's in the late 1940s, 
Stufflebeam and Skinkfield's Context-Input-Output-Output (CIPP) model emerged in the 
late 1960s. This model involves a new definition of evaluation different from that of the 
preceding authors: “evaluation is the process of identifying, obtaining and providing useful 
and descriptive information about the value and merit of the goals, planning, realization and 
impact of a given object, in order to to serve as a guide for decision making, solve problems 
of responsibility and promote understanding of the phenomena studied” (Stufflebeam and 
Skinkfield, 1987: 183). 

The above definition exposes the evaluation not as a test but as a process. Thus, the 
model under consideration assumes that what is essentially important in an evaluation is not 
to demonstrate but to improve. For this reason, the use of the CIPP model is conceived to 
promote the development and help the directors and responsible personnel of the institution 
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to obtain and use continuous and systematic information. 


The contents that this model evaluates are structured in the following dimensions: 
a) Context evaluation (C), whose objective is to determine the institutional context, the 
target populationistudy, diagnose needs and problems, etc.; b) the evaluation of Input (1), 
whose objective is to identify the capacity of the system, strategies, planning of procedures, 
budgets and programs; c) the evaluation of the Process (P), which identifies or predicts 
during the process the defects of the planning, of its realization; and, d) the evaluation of 
Product (P), which proposes to collect descriptions and judgments about the results and 
related to the objectives and information of the other three previous evaluations, interpreting 
their value and merit. And, the instruments used in the context of this model are very varied: 
systems analysis, inspection, hearings, interviews, tests, inventories, analysis of human 


and material resources, visits, etc. 


Another option linked to the efficientist-behaviorist models is the model of the figure 
for the educational evaluation of Stake (1975).!º In this model that was developed at the end 
of the 1960s, the figure is conceived as the image of educational evaluation, and it is built 
on what Tyler thought of the evaluators as they must compare the desired and observed 
results, highlighting the importance of analyzing the achievement of results based on the 
objectives set. 


In addition, Sake's method encourages evaluators and educators to pay more 
attention to the whole set of evaluation that includes description (description of the 
evaluation, its complexity and its importance), judgment (a evaluation is meaningless until a 
judgment is made), scheme for data collection (background information relevant-, teaching 
transactions -encounters of students with all people-) and results (what is achieved), 
analysis of congruence (have the purposes been fulfilled?) and contingencies, identification 
of relevant norms and often opposed and, the multiple uses of the evaluation. 


3.1.8 Humanistic, Phenomenological or Subjectivist Models 


The so-called humanistic, phenomenological or subjectivist models are oriented to 
consider a growing concern for individuals, for their activities, for the processes and for the 
phenomena they produce. And the cultural models referred to by De Miguel (1984) must be 
included here, since, when concerned with the processes, they take into account variables 
such as: the values of people, their beliefs, the cultural substratum of their own lives, their 
attitudes towards the environment etc. In addition, its purpose is to improve educational and 
learning processes with a results-oriented perspective, and they use both qualitative and 
quantitative instruments, the evaluator collaborating with those involved in the evaluation 


16 For some authors such as Worthen and Sanders (1987), and Colás and Rebollo (1993), this author would be inscri- 
bed within a current more of a humanist, subjectivist type, since he broke with many of the Tylerian topics when develo- 
ping in the early 70s a new set of concepts and which he introduced as “Respondent Evaluation”. 
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processes, establishing a climate of intercommunication, responsibility and reflection aimed 


at personal and professional improvement. 


Among the options of these types of models, Owens" (1973) conflicting evaluation 
method, Wolf's (1975) judicial model, Scriven's (1967) customer service model, and Eisner's 
(1981) artistic criticism proposal stand out.). 


Owens's (1973) contrasting method of evaluation was developed in the late 1970s, 
already marking a step toward the end of the Tylerian tradition. Its ultimate goal was to 
improve decision-making, for which it was necessary for the method to provide those who 
make those decisions with information so that they could develop projects and judge the 
expected and unforeseen results so that they could act. 


This method was developed to provide an explanation for those variables that are 
less quantifiable or tangible (school climate, student attitudes, etc.) and that are normally 
rejected. Opposition proceedings are developed as an administrative hearing intended to 
judge the merits of a case involving two opposing parties (two groups of evaluators, for 
example) so that the best decisions can be made in an open and participatory manner., 
by being able to intervene those people who are interested in the evaluation process 
(Stufflebeam and Shinkfield, 1987: 296). 


In this option, Owens proposes seven main ways in which the adversarial model can 
be carried out to suit the purposes of an educational assessment: Explorations of the values 
of the existing curriculum or a new one (to be developed through an open debate on the 
relevance or not of a specific curriculum); selection of new textbooks (a hearing could be 
held between teachers who are for and others against certain textbooks); estimation of the 
congruence between an innovation and the existing system (decision making can be oriented 
towards the relevance of that innovation); disclosure of the different interpretations made 
by different representatives of the same data (in the school there are teachers, students 
and administrators); decision making can be collective, as if it were a jury; information to 
teachers, directors and administrators, resolution of disputes about employment contracts; 
and, finally, arriving at the decision that must be put into practice. 


For the author in question, these contrasting methods may be relevant for the 
application of the models developed by Stufflebeam and Stake. And, the evaluator continues 
to be considered as external but who serves as a mediator in the debate between those 


involved in the evaluative discussion. 


The judicial model of Robert Wolf (1973) constitutes one more option in the 
humanistic, phenomenological or subjectivist schemes. This option was developed from 
that of Owens, and constitutes a variant that offers the form of trial based on a court before 
which arguments and justifications are offered to facilitate an agreement. 


For this author, the solution to conventional evaluation does not lie in collecting a 
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large amount of technical data but in clarifying these data and their information. Evaluators 
can focus on a certain problem and have the testimony of other people and other evaluators; 
they can analyze the methods by which data is collected and analyzed, in order to obtain a 


more balanced point of view. 


In the development of this proposal, Wolf proposes a sequence of four phases or 
stages, namely: a) The stage that proposes programs mainly through interviews; b) the 
stage that selects the problems. Problems are narrowed down until those that serve an 
audience are identified. A group of researchers will decide the importance of each problem; 
c) the argument preparation stage: the evaluation teams then prepare the formal arguments; 
and, d) the stage of the hearing with two parts, a previous one in which the arguments of 
both teams are presented and the rules of procedure are established and another that is 
the hearing itself. 


On the other hand, there is the so-called customer service model of Scriven 
(1967). For some authors such as Farley (1985) and Colás and Rebollo (1993) this model 
is considered within the objectivist models; Medina and Villar (1995) include it within the 
phenomenological models; Stufflebeam and Shinkfield (1987) incorporate it within the 
holistic models, so it seems that there is no unanimity when it comes to classifying this 
model option. This time, however, it will have to be placed between the goal-focused and 
the purely holistic. 


This approach arose as an opposite response to Tyler's models based on achieved 
objectives, giving more importance to those unforeseen achievements than those that 
appeared as foreseen. Developed by Scriven in 1967, it focused more on analyzing how 
educational programs affect people than on the goals, that is, it rested on the analysis of 
the client's needs, passing the objectives to the background and making a value judgment 
well informed about an object based on the evidence accumulated from its comparison with 
other objects, in order to satisfy consumer needs. 


In the comparative evaluation proposed by Scriven, moreover, it is encouraged that 
the evaluator does not know what the foreseen objectives are so that they do not hinder 
the evaluation of other aspects, which would make it incomplete and erroneous (what has 
been called evaluation without goals).). However, this process is reversible, since once all 
the unexpected effects have been analyzed, the process based on the goals can begin to 
see if they have been achieved. And, finally, two cardinal functions are given to evaluation: 
the formative one -which helps to develop the evaluative process by helping to plan and 
perfect anything- and the summative one -which calculates the value of the object once 
it has been developed, providing judgments about the extent to which goals validly reflect 
assessed needs. 


And, in Eisner's artistic criticism proposal (1981), the conception of evaluation starts 
from a humanistic-critical line because the fact that people can express their opinions 
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means that there is a more balanced, complete and creative vision of the subject. object of 
evaluation. For this reason, its purpose is that the people involved in evaluation processes 
have interpretive and evaluative attitudes of the educational process and its context. For 
this, the evaluator who acts as a promoter of opinions of teachers and students, which 
allows him to know the educational reality, interpreted and judged taking into account the 
context, the educational processes, the emerging processes, the relationships between the 
different processes, seeking the critical assessment that leads to the issuance of value 
judgments. 


3.1.9 Holistic Models 


Around the 1960s, the models ascribed to this typology began to develop, a situation 
that coincided with the appearance of the interpretive paradigm, of an eminently qualitative 
nature. Holistic models have a global vision of the different components of the educational 
center, which requires taking into account the opinions, values, beliefs and aptitudes of all 


those involved in it. 


From a holistic perspective, evaluation is conceived as a process of understanding 
and evaluating processes and results. In this option, the evaluator is a cooperator with the 
evaluation participants. 


Among the options of these types of models, the respondent evaluation of Stake 
(1983), the illuminative evaluation of Parlett and Hamilton (1983) and the holistic evaluation 
of McDonald (1983) stand out. 


In the first place, as a result of the evolution of the model of the figure discussed 
in advance, Stake developed an evaluative alternative in the mid-1970s that breaks with 
the Tylerian tradition. This method emphasizes the development of learning, didactic 
transactions, data for judgments, holistic reporting, and assistance to educators. 


The purpose of this type of evaluation is to help teachers, students and administrative 
staff understand the problems, strengths and weaknesses of educational programs. To 
do this, the evaluator must refer to people's opinions without judging them, collecting 
information throughout the process and responding to those problems or questions that 
arise. To achieve this, the plans will be open and flexible, tending more towards description 
than assessment.” 


Stake identifies four parts in the substantial structure of the respondent evaluation, 
namely: a) The problems (areas of disagreement, issues to be resolved); b) a scheme for 
data collection (which will come from the model in the figure); c) human observers (as the 
17 The methodology will first include observation to later intervene (naturalistic research) introducing techniques such as 
case studies, purposive sampling, narrative reports, etc., using, if necessary, other methods. And, communication with 


the client will occur naturally, frequently and informally, since the purpose is to continuously locate issues that concern 
them. 
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best research instruments); and d) validation (through a lot of information that allows you 
to have a vision of the object under study). In addition, the method requires a large amount 
of observation, in a process in which linear phases do not exist and where continuous 


information is the most important. !8 


Secondly, in contrast to the conventional evaluative models of experimental or 
psychometric traditions of very limited scope, other methods arise, whose origin is in social 
anthropology, which want to analyze reality as a whole, using description and interpretation 
(versusto assessment and prediction). Itis what has been called the illuminative methodology. 
lts authors, Parlett and Hamilton (1983) promote the illumination of the different components 
of the evaluation to describe them and make them more comprehensive in relation to the 
context. 


Thus, the adoption of an illuminative evaluation supposes understanding two main 
aspects: the description of the teaching system and the learning environment as the 
environment in which students and teachers work. It is a method that does not attempt 
to manipulate, control or eliminate situational variables, but instead, using a combination 
of techniques, attempts to analyze and understand the relationships between beliefs and 
practices and between organizational models and those of individuals. emphasizing the 
observation of classrooms, interviews with teachers and students, questionnaires and tests, 


and documentary and background sources.!º 


And, thirdly, there is the holistic evaluation of McDonald (1983), a scheme that 
proposes that the evaluation must take into account and must be a matter of concern for 
all possible data and the context, given that human action within institutions is exposed 
to a multitude of variables that influence it. The holistic method supposes analyzing each 
educational act within the totality, involving the protagonists in the interpretation of reality 
in which conceptions, beliefs, values and interests appear that must be explained through 
collaborative participation in the debate, becoming in the content of the evaluation. 


And, from this naturalistic vision, evaluation aims to facilitate and promote change, 
developing analysis and assessment activities (McDonald, 1983; and Medina and Villar, 
1995).20 


18 This functional structure of the evaluation responds to the form of evaluation clock: Twelve o'clock, talk to customers 
and interested audiences; one: identification of the scope of the problem among the interested parties; both: identify 
activities; all three: discover the purposes of the evaluation and the interests of the people; all four: analyze issues and 
problems; the five: identify the necessary data to investigate the problems, the six: select the observers, judges and 
instruments; seven o'clock: observe the history of the transactions and the proposed results; the eight: develop themes, 
prepare descriptions and study concrete cases; nine o'clock: validation; the ten: schematization of the information for the 
hearings; eleven o'clock: gather formal reports if any. 

19 lts methodological strategies emerge from negotiation processes, and the evaluator must promote debate among the 
participants so that opinions, interpretations and value judgments emerge from there, acting as a cooperator. 

20 In addition, the evaluator offers information and search and contrast procedures through dialogue, discussion, search 
and analysis; and promotes interpretation using qualitative techniques (Colás and Rebollo, 1993). 
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41 EVALUATION MODELS FOR PUBLIC UNIVERSITIES 


In recent times, interest in quality in University has led countries to generate a 
whole series of quality control systems for university institutions, known under the name 
of institutional evaluations (Mora, 1991). And these institutional evaluation schemes have, 
finally, a double objective: On the one hand, to allow the university to know the quality of its 
activities and define strategic action plans to improve its activities; and, on the other hand, 
offer their financiers (students and their families, companies and institutions interested in 
University and research, Public Administrations) objective and reliable information on the 
level of quality achieved by each institution (Pérez and Salinas, 1998: 161). 


Evaluations can be classified, depending on who performs them, under two types: 
Internal and external evaluations. The main difference between them lies in who is the agent 
that carries out the evaluation. In the first case, the members of the institution themselves, 
while in the second they are people from a specialized agency or an external commission. 


Currently, mixed evaluations are frequent in which both perspectives are combined 
so that the evaluation of an institution is the combined result of a self-evaluation and an 
external evaluation. Likewise, the evaluation can be thematic, when it refers to a study 
program or career. specific or to a set of them belonging to the same scientific-teaching field, 
in one or several University Institutions (HEIs) or universities, or global, when it includes one 
or several HEIs or universities, globally considered, and include all degrees, departments 
and services that the HEI or university provides. 


Given that education in HEIs encompasses five well-differentiated areas: teaching 
or teaching, research, extension, dissemination and administration or management, in the 
evaluation of HEIs or universities these areas or types of activities are usually considered 
by separately, although the existence of complementarity and interdependence between 
them seems evident. 


From another point of view, institutional evaluations can be classified according to 
the purpose pursued by the evaluation: a) Evaluations for the control of the institution by 
government institutions; b) evaluations to know the real situation of the institution with the 
aim of improving quality, promoted by the institution itself or by other instances, including 
government (Mora, 1991: 53). 


Another distinction can be made according to the degree of obligation with which 
the institution must respond to the results of the evaluation. Evaluations can be imperative 
when the HEI or university has to faithfully follow the results of the evaluation, which is what 
usually happens with most government evaluations. In other cases, the evaluations have 
the objective of recommending standards, not imposing criteria, which is why they are called 
regulations. 


There are also consultative evaluations that aim to find out the situation itself and 
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find solutions that can help improve it, usually requested by the university or carried out 
by it (Kogan, 1986). Table 1Il.1 summarizes these different types of evaluations for HEIs or 
universities. 


These different classifications can be synthesized in two approaches: One focused 
on aspects of efficiency, economic and control factors, which refers to the optimization of 
resources, and another focused on organizational aspects of the teaching process, which 
refers to the verification of the degree of achieving the objectives and that is oriented towards 
improving quality (Buendía and García, 2000: 221). And, the adoption of one approach or 
another will give rise to different models of institutional evaluation, which must be identified 
with the reality of each university system. 


Table Ill.1. Types of Evaluation in HEIs 


Internal assessment or self-assessment (from the HEI or 
university itself). 

External evaluation (commission of experts). 

- Thematic evaluation (study program or career). 

- Global evaluation (of one or several HEIs). 

- Evaluation of teaching activity. 

- Evaluation of research activity. 

Types of activities - Evaluation of the extension activity. 

- Evaluation of the dissemination activity. 

- Evaluation of the administration or management. 
- Bureaucratic control. 

- Quality improvement. 

- Imperative. 

Mandatory - Normative. 

- Advisory. 

Source: Own elaboration with data from Mora (1991: 54). 


Department that carries out the 
evaluation 


Project scope 


Object or purpose pursued 


Finally, among the evaluation models for PU, in consideration of the preceding 
guidelines, accreditation, program review, performance indicators and external evaluation 
based on expert judgments are identified. In the following spaces, it will be pointed out in 
this regard. 


4.1 The Accreditation Model 


Accreditation is a process by which an educational program or institution provides 
information on its activity and achievements to an external committee that independently 
evaluates such information in order to issue a public judgment or based on explicit 
standards on the value and quality of the institution or program. According to El-Khawas 
(1993), accreditation places the emphasis on inputs as a guarantee of the quality of outputs, 
is oriented towards efficiency and has a high component of summative evaluation. 


Accreditation implies the evaluation of an educational institution or program by a 
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group of external experts. To do this, the institution or educational program requests its 
evaluation by providing a self-study to the group of external experts who evaluate it and 
issue a public judgment based on pre-established standards. 


And, there are two types of accreditation, one that accredits HEIs globally and 
another that accredits programs of a specific professional type. Accreditation processes 
tend to be a bureaucratic process of diminishing interest due to the fact that institutions have 
internalized quality mechanisms, incorporating them into normal practice.?! 


4.2 The Program Review Model 


For its part, the review of programs constitutes an evaluative approach that 
encompasses all the programs of an institution that generally places emphasis on processes 
as a strategy for constant quality improvement. lts orientation towards improvement and 
development contrasts with the accreditation approach, aimed at verifying compliance with 
minimum standards in a professional and/or disciplinary field (Barak and Breier, 1990). 
The fact that the evaluation initiative corresponds to the institution makes the staff of the 
program and/or the institution the primary agents of the review. It is, in short, the origin of 
internal evaluation or self-evaluation. 


Program review is an evaluative approach that encompasses all of an institution's 
programs and places emphasis on processes, as a strategy for constant quality improvement, 
which distinguishes it from the accreditation approach, aimed at verifying compliance with 
the minimum pre-established standards. The fact that program review has been applied to 
different types of programs and institutions, also including an accountability orientation, with 
the participation of external agents, has led to the use of the term institutional evaluation for 
this type of evaluation (Rodríguez, 1997). 


Unlike evaluation, program review is not confronted with previously established 
criteria, but only aims to describe and assess the history of the unit or program, its current 
performance, as well as its future plans (Holdaway, 1988). The two basic purposes attributed 
to itare: To serve as a basis for the allocation and reallocation of funds and fundamentally, 
the improvement of programs. 


Some of the aspects to take into account when planning the review of programs 
could be (Tejedor, 1997): Separate the review of the material and organizational elements 


21 For this reason, in the USA it has recently been proposed to convert accreditation processes into assessment audit 
processes, that is, into quality meta-assessment processes (Mora, 1998). 

22 Although there is no universally accepted procedure for implementing program reviews, Holdaway (1988) and Young 
(1983) point out some of the steps that are common to all reviews: Determine who is requesting the review and what they 
are their prospects; identify the specific purposes of the review; identify the main issues on which to focus attention; limit 
the extension field to revision; identify the process to be used and the time periods that will be required; design methods 
to collect and analyze numerical data, information and opinions. In addition, Kells (1988), distinguishes the following 
phases for the implementation of a program review: Carrying out a self-study by the professionals of the program that 
is going to be reviewed; external review by experts from the same scientific field as the revised program; review of the 
self-study and the experts” report by a group of professionals from the institution itself, and, as a consequence of the 
results of the three previous steps, an agreement is reached with the institution's management. 
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from the personal elements, which require special attention ?º and differentiate the individual 
and institutional perspectives.?* 


4.3 The Performance Indicators Model 


Performance indicators deal with objective, usually quantitative, measures of an 
institution or an entire University system (Ball and Halwachi, 1987). They provide operational 
information on the functioning of the institutions and on their effectiveness. 


The use of performance indicators of university institutions has its origin in external 
evaluations of a governmental nature and with the purpose of distributing funds. The 
three categories of indicators according to Cave et al. (1997) are the following: a) Simple 
indicators, formulated in absolute terms, are aimed at giving an objective description of a 
situation or process (management statistics); b) performance indicators, which imply a point 
of reference, that is, they are of a relative nature; and c) general indicators, which refer to 
statistics, opinions or survey results. 


The use ofthis type of indicators has been and continues to be a source of discussion. 
The most recurrent criticism refers to the possibility that such indicators can be constructed. 
These difficulties have led to the establishment of partial performance indicators, but 
optimizing the parts does not necessarily mean optimizing the system as a whole (Sizer, 
1979). And, despite this problem, Sizer (1982) defends the use of these indicators since 
they help to quickly detect the trends of changes in the needs of society, in the preferences 
for certain types of studies, and in the reallocation of means. 


4.4 The External Evaluation Model based on Expert Judgments 


External evaluation based on external judgments (peer review) is a type of evaluation 
that is located in the context of either the request of an HEI or university for the analysis of 
one or all of its academic units and services, or by the imposition of the IES system itself. 
The origins of this type of evaluation are located in the program accreditation committees, 
in the evaluators of scientific journals and in the advisors of research funds (Rodríguez, 
1997: 191). 


Also known as peer review, this type of evaluation can be requested by an HEI for 
the analysis of one or all of its academic and service units, or it can be imposed by the 
university system itself (Rodríguez, 1997). The external evaluation is carried out by a Visiting 
Committee, made up of experts in the area, who are at the same time colleagues. This 
Visiting Committee fulfills a double function: On the one hand, improvement, related to the 


23 The aspects to be addressed would be: Quality (of the teaching staff and students; of the curriculum; of the support 
services; and of the administrative services), the need for the program (for the university itself and for society), current 
and foreseeable demand for students and society for these graduates, and costs and benefits. 

24 From the individual point of view (review of results -assessment of the students" learning-; and the teaching evaluation 
of the teaching staff or the evaluation of their research productivity) and from the institutional point of view (goals and 
objectives of the university, demand of the degree, teaching plan, student performance indicators). 
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feedback that the committee provides to the teaching staff, and on the other, accountability, 
through the public report that it issues and that it makes known to society. on the quality of 
the institution (Vroeijensti and Acherman, 1991). 


This type of evaluation has been criticized because the issuance of a judgment by 
the Committee of Experts includes a component of subjectivity. However, as Mora (1991) 
points out, the evaluation system that makes use of indicators and peer reviews may be the 
most appropriate. The advantage offered by the use of indicators to establish criteria for 
the rational allocation of resources seems unquestionable, while the essential advantage 
offered by peer review is that it makes it possible to take the first steps in the establishment 
of an institutional evaluation system. 


51 INSTITUTIONAL EVALUATION AND IMPROVEMENT OF EDUCATIONAL 
QUALITY UNIVERSITY 

As has been seen, in recent times the evaluation of the quality of university institutions 
has become a priority issue. This interest in evaluating the quality of university institutions 
must bring about a change: the transition from a bureaucratic vision of the system to a more 
autonomous vision. 


And this greater autonomy implies a greater and better evaluation of the institutions 
and a different perspective that supposes, in addition to a verification of the functioning of 
the system, a method to improve it (Húfner and Rau, 1987). 


Faced with this situation, governments demand accountability from universities, in 
order to demonstrate that both the decisions made and the actions carried out are aimed at 
improving quality (Rodríguez, 1997). All of this, together with citizens” demands for quality in 
all production processes and services, economic competitiveness, and relative financial and 
student stability, seem to indicate that the time has come to worry about improving quality. 
of university institutions. 


This way, it is observed how throughout the last decades the governments of 
developed countries have been concerned with achieving different goals for the improvement 
of the quality of university institutions. Thus, while in the 1960s, the priority objective was 
to accommodate the largest number of students who applied to study at the university, in 
the 1970s, as a result of public spending on universities and the increased democratization 
in access to Universityal levels, the concern was to establish a management system for 
the university process, which would assess the effectiveness and efficiency of university 
institutions (Mora, 1998). However, it was really in the 1980s and 1990s that priorities began 
to shift towards improving the quality of service provided by our university institutions. 


Although the concept of quality has been accepted by various authors, which 
postulates, on the one hand, the adjustment to social needs, in addition to compliance with 
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institutional norms and effectiveness in achieving the established goals, it must be borne in 
mind that there are still serious difficulties in defining quality. 


In this regard, Álvarez (1998) states that today the competition is so enormous that 
the product is going to become an offer, that is, it is going to be, above all and above all, a 
response to the needs and expectations of the client, considered not only as consumer, but 
as a satisfied user of the service. Therefore, quality becomes the result of careful attention 
to processes, good raw materials, well-planned work and the way in which the product or 
service is delivered to the customer. 


And another of the outstanding characteristics of quality is also its complexity to 
be evaluated. However, this has not been an impediment for researchers from different 
countries to try to evaluate it. Thus, the approaches used to assess the quality of a university 
institution can be framed in the categories proposed by George (1982) and Astin (1985): 
a) Quality as prestige; b) quality as availability of resources; c) quality through results; d) 
quality for the content; and e) quality for added value. 


In the first place, quality understood as prestige refers to the academic and social 
reputation of some institutions, distinguishing them from others. From this approach, the 
quality of a university institution is assessed through expert judgments, so it cannot be 
objectively evaluated. And, the main criticism of this approach lies precisely in its subjectivity. 


On the other hand, quality conceived as the availability of resources weighs the 
ability to possess human, economic and physical resources. And the criticism leveled at 
this approach is that it does not always coincide that a large institution, because it has more 
resources, is of higher quality. 


Thirdly, the quality appreciated through the results indicates that a university 
institution is successful due to the success of its graduates. And, the biggest criticism of 
this approach is that the success of the graduates may be due more to the context or to the 
student himself than to the institution. 


In considering quality for content, it can be mentioned that from this approach the 
quality of university institutions is valued based on the concepts involved in the learning 
process (curriculum, pedagogical system, etc.). However, the problem lies in the difficulty of 
evaluating these concepts. 


And when quality is seen for added value, it is assumed that the highest quality 
institutions are those with the greatest impact on knowledge, for example behavioral change 
or career development. However, the way to isolate the influence of the center from other 


social forces is an unanswered question. 


Currently, as Mora (1991: 35) points out: “The problem of the optimal approach 
to quality has not been resolved, the search for quality through measures such as those 
indicated has little to do with improving the system as a whole and with the quality of the 
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educational program. However, all of them together, duly weighted, must give a certain idea 
of the quality of an institution”. And, in any case, says Tejedor (1997: 413): “lt seems obvious 
that a university institution can only achieve a reasonable level of quality when the human, 
financial and physical elements, teaching and research, organization and management, are 
appropriate for the purposes pursued by the institution”. 


And, in some way, all these approaches participate in considering a quality university 
institution when itis pertinent, efficient and effective. The relevance must lie in the existence 
of congruence between the expectations of the context and the institutional offer (external 
dimension) and the congruence between the teleological platform of the institution and the 
resources and procedures that are arbitrated to achieve them (Villarroel, 1999). 


Efficiency, for its part, can be defined as an optimization of resources in achieving 
goals. And, finally, effectiveness has to do with the achievement of objectives and goals. 
However, as Villarroel (1999) points out, a difference is usually made between effectiveness 
and efficiency. The first would cover the achievement of objectives and goals in the 
institutional sphere, and effectiveness would rather refer to the impact that the institution 
achieves with its products. 


Therefore, if university quality is considered this way, the institutional evaluation 
would be aimed at evaluating the relevance, efficiency and effectiveness of said institution. 
Most authors understand that the evaluation of the quality of a university can only be done 
globally, attending to these concepts and collecting both the inputs, the products and, above 
all, the processes that are developed in it. 


Finally, the process for said evaluation, as it is being carried out in the countries of 
Europe and America, mainly, is divided into two main phases: Self-assessment and external 
evaluation. This way, the process is regulated by the institution itself and, in turn, the internal 
review acquires credibility when it is contrasted by external agents. And, in this sense, the 
institutional evaluation allows responding, in principle, to two types of requirements: On the 
one hand, it is a basic instrument for the internal improvement of the quality levels of the 
university; and on the other hand, it facilitates accountability and analysis of the adequacy 
of the services provided by universities to the needs and demands of society (Jofre and 
Vilalta, 1998). 
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EVALUATION OF THE QUALITY OF UNIVERSITY IN MEXICO 
1] UNIVERSITY IN MEXICO 


1.1 Characterization of the Mexican National Education System and its Social, 
Political and Economic Context 

ltwasinthe second halfofthe 20th century, when Mexico underwent an industrialization 
process that had repercussions on its economic and social structure. Then, the urbanization 
scheme accelerated, so HEIs have witnessed the assessment of education as an eminently 
urban phenomenon, and this has been so due to the registration of unplanned growth of 
cities and existing imbalances in the regions. (ANUIES, 2000). 


From 1900 to 2000, the nation underwent a profound change in the settlement 
patterns of its population. In today's Mexico, this problem has accelerated as a result of 
the economic strategy that the country has adopted in recent years, since joining the GATT 
-now the WTO- in 1986, and the OECD in 1994. Thus, The nation has joined international 
markets through its participation in trade agreements with countries in North, Central and 
South America and Europe.,? Therefore, there are now eleven free trade agreements 
negotiated by Mexico, which gives the country safe and preferential access to the markets 
of 32 countries that represent approximately 860 million consumers. 


Therefore, Mexican society is more urban and modern today. Although, at the same 
time, regions of the country coexist that have not benefited from economic growth. The 
adaptation of people to the challenges that this new model implies has accentuated the 
inequality of Mexicans. And on the political issue, the country has experienced an expansion 
of its democratic life, and that forms an essential part of the political transition that the 
country has experienced since 2000. The developments in this area have consolidated 
important achievements, such as the alternation of governments at the state and federal 
levels; as well as the strengthening of political parties and associations. 


In the social sphere, changes of the first order have also been generated, as is the 
case of the forms of association that were restricted to union and sector groups. Now, there 
are multiple initiatives of civil society organization. And social barriers have grown and in 
some cases accentuated by economic policy and income polarization between different 
social sectors. 


Also, within the breaking of traditional schemes, which Mexican society faces on a 


1 In 1910, one in ten inhabitants lived in one of the 33 cities of that time. Currently it is estimated that seven out of ten 
inhabitants live in one of the 372 urban centers of the country. (Cf. Ministry of Public Education, 2001: 28). 

2 Around 1992, the Economic Complementation Agreement with Chile entered into force; in 1994 the North American 
Free Trade Agreement; in 1995, free trade agreements were launched with Bolivia, Costa Rica and the so-called “Group 
of Three” with Colombia, Venezuela and Mexico; in 1998 the Free Trade Agreement with Nicaragua entered into force; In 
2000, free trade agreements with Israel and the European Union entered into force, with the northern triangle made up 
of El Salvador, Guatemala and Honduras, with the European Free Trade Association, made up of Iceland, Liechtenstein 
and Switzerland. (Cf. Foreign Trade Information System, 2008). 
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daily basis, the transformation of the role of women stands out. In addition, the presence 
of the youth population demanding employment and greater social participation, forces the 
different governmental levels to create new political strategies in the face of the challenges 
of Mexican society in the 21st century. 


Undoubtedly, of these solutions, HEIs have a very important role, since they will train 
qualified personnel who will lead the country's economic, political and social development. 
Therefore, HEIs are obliged to undertake a set of rearrangements and reforms to respond 
to the demands of Mexican society and today's world. 


1.2 The National Education System in Mexico and the Location of Education 
Levels 

Within government policies, education is appreciated as a fundamental pillar for life 
and progress in Mexico. Thus, in the National Education Program 2001-2006, the allusion to 
the topic is expressed as follows: “It is imperative to rethink the tasks of Mexican education 
so that it effectively contributes to building the country we want: the nation fully democratic, 
with a high quality of life, dynamic, proudly faithful to its roots, multi-ethnic, multicultural and 
with a profound feeling of national unity, to which the National Development Plan 2001-2006 
adheres; a country in which extreme social inequalities have been reduced and the entire 
population is offered opportunities for development and coexistence based on respect for 
the law and the real exercise of human rights, in balance with the environment” (SEP, 2001: 
16). 

In the same sense, for its part, the Education Sector Program 2007-2012 establishes 
that: “Education is the basis for the progress of nations and the well-being of peoples. 
[...] [And,] In the knowledge society, the competitiveness of countries depends, to a large 
extent, on the strength of their educational systems and their ability to generate and apply 
new knowledge. Mexico must make education, science and technology the pillars of its 
development. In them is the solution of the most pressing national problems; the increase in 
the quality of life of the population depends on them” (SEP, 2007: 7 and 10). 


In addition, article 3 of the Political Constitution of the United Mexican States states 
that education must be provided by the State and will have to harmoniously develop all the 
faculties of the human being, and foster in him, at the same time, love to the Homeland and 
the conscience of international solidarity, in independence and in justice. Thus, in section 
Il, it is said that education will be democratic, considering democracy not only as a legal 
structure and a political regime; if not as a way of life based on the permanent economic, 
social and cultural improvement of the people. 


Therefore, the legal framework of educational policies, and the National Education 
System (SNE) in Mexico define the fields of educational actions in the country, in order 
to achieve a good quality of it, a necessary condition for development. fair and balanced 
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national (SEP, Op. Cit.: 7 and 8). In order to understand the link of the SNE, in compliance 
with the constitutional precepts and the current demand, both in qualitative and quantitative 
terms, itis necessary to review the articulation of the different levels that constitute it. 


Thus, the SNE in Mexico is segmented into: Basic education, upper secondary 
education and University. Next, reference will be made, from three sections, to these levels 
of the education system. 


1.3 The National Education System in Mexico and University 


In the casuistry of Mexico, University (ES), registered in the SNE, includes studies 
after those of the EMS; and is taught both in public and private or private institutions. In 
addition, the ES has as its purpose the training of people at the higher technical university 
or associate professional levels, bachelor's, specialty, master's and doctorate. The activities 
of University Institutions (IES) vary according to the type and mission of each one. And, 
among these activities are: Teaching, scientific, humanistic and technological research; 
technological studies and the extension, preservation and dissemination of culture. 


According to the Federal Government, University is recognized as the pillar of 
Mexico's development. Thus, in the National Education Program 2001-2006 it is referred to 
as follows: “University is a strategic means to increase the human and social capital of the 
nation, and the individual and collective intelligence of Mexicans; to enrich culture with the 
contributions of the humanities, arts, sciences and technologies; and to contribute to the 
increase in competitiveness and employment required in the knowledge-based economy. It 
is also a factor to promote the growth of the national product, social cohesion and justice, 
the consolidation of democracy and national identity based on our cultural diversity, as well 
as to improve the distribution of the population's income” (SEP, 2001: 183). 


In the same logic as its immediate predecessors, the current federal public 
administration, within its government program, has conceived that the mission of University 
today consists of “(...) strengthening the equity, quality and relevance of this type of 
education to train creative citizens and professionals, as well as scientists and technologists 
committed to their country, in order to turn University into a true engine of development to 
enter with advantage in the emerging knowledge economy” (Presidency of the Republic, 
2007b: 278). 


1.4 Configuration of the University System in Mexico 


1.4.1 General 


In the National Education System, University constitutes the last structural level; 
and its offer consists of professional education and specialization. Professional education 
corresponds to bachelor's and higher technical degrees, and the second to postgraduate 
degrees. The first is intended to prepare students in some discipline or specific knowledge 
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for the authorized and professional exercise of a specific activity. And the postgraduate 
degrees grant degrees of specialization in more limited subjects. 


Regarding the duration, professional studies consist, depending on the academic 
plan, between 3 and 6 years. They are mostly attended by students who are 18 years 
old at admission. For their part, postgraduate programs last, depending on the degree of 
specialization, between 1 and 4 years. 


Finally, within the SNE, the University System (SES) is located, that is, professional 
education and specialization. Currently, the SES is made up of more than 1,500 public and 
private or private institutions, offering educational programs for higher university technicians 
or associated personnel.,? bachelor's degree,“ normal education,* and postgraduate* (SEP, 
2001: 186). 


1.4.2 Regulatory framework 


The SNE has its mandate in the national regulatory framework related to education, 
which is made up of the Political Constitution of the United Mexican States and the General 
Education Law. 


Thus, the third article of the Magna Carta establishes the basic norm in education. 
And, specifically, in sections V, VI and VII, what is related to University is outlined: “(...) V. 
In addition to providing preschool, primary and secondary education, indicated in the first 
paragraph, the State will promote and attend to all the educational types and modalities, 
including University, necessary for the development of the nation, will support scientific 
and technological research, and will encourage the strengthening and dissemination of our 
culture. SAW. Individuals may provide education in all its types and modalities. In those 
established by law, the State will grant and withdraw recognition of official validity to studies 
carried out in private schools. VII. Universities and other University institutions to which the 
law grants autonomy, will have the power and responsibility to govern themselves; they will 
carry out their purposes of educating, researching and disseminating culture in accordance 
with the principles of this article, respecting the freedom of academics and research and the 
free examination and discussion of ideas; they will determine their plans and programs; The 
terms of entry, promotion and permanence of their academic and administrative personnel 
3 The university senior technician or associate professional category offers two-year degrees in the areas of production 
and services, and leads to university senior technician degrees. 68% of enrollment at this level is focused on 44 relatively 
recent technological universities in the country. Currently this type of university offers 25 careers. The rest of the enroll- 
ment for this degree is taken care of in private institutions (SEP, 2001: 186). 
4 Undergraduate studies include careers with a minimum duration of four years. lts enrollment amounts to 1,664,384 
students, of which 68.6% are enrolled in public institutions, and 31.4% in individuals (Ibídem: 186). 
5 This higher education degree offers undergraduate and postgraduate programs for the training of teachers of basic 
and specialized education. 60% of enrollment is attended by public institutions, and 39.9% by private schools (Ídem). 


6 Postgraduate studies include specialty, master's and doctorate studies. Currently, 21.9% of the enrollment is in the 
specialty, 71.1% in the master's degree and 7% in the doctorate. 59.4% of enrollment is attended by public institutions 


and 40.6% by private ones. (Ibídem: 187). 
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will be established, they will be regulated by section A of article 123 of this Constitution, 
in the terms and with the modalities established by the Federal Labor Law, in accordance 
with the characteristics of the of a special job, so that they are consistent with autonomy, 
academic and research freedom, and the purposes of the institutions to which this fraction 
refers” (Political Constitution of the United Mexican States, 2007: 6-8). 


Regarding the General Law of Education, this establishes in its first article that “(...) 
The educational social function of universities and other institutions of University referred 
to in section VII of article 3. of the Political Constitution of the United Mexican States, will 
be regulated by the laws that govern said institutions” (General Law of Education, 2004: 4). 


For its part, the Law for the Coordination of University aims to establish the bases 
for the financing of University between the Federal Government, the states and the 
municipalities.” 

And, in the case of the PUSs, their laws are organic and issued by the corresponding 
Congress, from which the institutions generate their internal regulations. Technological 
institutes also have their internal regulations. The organic laws of the autonomous public 
universities establish the purposes, structure, form of government, administration methods, 
general provisions on their operation, rights and duties of students and professors. The 
PUs approve secondary regulations where the rules for admission of students, professors, 
promotion, as well as the powers, terms and limitations of their collegiate bodies, as well as 
the obligations and rights of their executive authorities, are established. 


In the same tenor, and taking up the General Education Law, it states in its ninth 
article that: “In addition to providing preschool, primary and secondary education, the State 
will promote and attend -directly, through its decentralized agencies, through of financial 
support, or by any other means - all educational types and modalities, including University, 
necessary for the development of the nation, will support scientific and technological 
research, and will encourage the strengthening and dissemination of national culture and 
universal” (General Education Law, 2004: 15). 


Likewise, the federated states issue their education laws adhering to the guidelines 
of the federal law. Other types of laws regulate complementary aspects of university life; 
such as is the case of labor relations in autonomous public universities. For example, article 
123 of the Constitution establishes that they will be governed by what is established in a 
special section of the Federal Labor Law. 


Autonomy is exclusive to the PUs and is the result of an Organic Law issued by the 


7 To this end, said law considers two councils as consultative bodies of the federal government: the Council for Nor- 
mal Education and the Council for the National Technological Education System. The Federal Government allocates 
resources to public institutions of higher education, in accordance with certain conditions such as institutional planning, 
academic improvement programs, administrative improvement, and priorities that stand out at the time. In relation to 
the foregoing, article 22 of said law establishes that, “(...) The income of public institutions of higher education and their 
property will be exempt from all types of federal taxes” (Law for the Coordination of the Higher Education, 2006: 19). 
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Federal Congress (in the case of federal public universities: the Autonomous University 
of Mexico, and the Autonomous Metropolitan University), or by the corresponding State 
Congress, as is the case of the autonomous universities of the United States.º 


Finally, accordingtothe General Directorate of University ofthe SEP, there are currently 
34 autonomous institutions in the country, namely: Autonomous University of Aguascalientes; 
Autonomous University of Baja California; Autonomous University of Baja California Sur; 
Autonomus University of Campeche; Autonomous University of Carmen; Autonomous 
University of Coahuila; University of Colima; Autonomous University of Chiapas; Autonomous 
University of Chihuahua; Autonomous University of Ciudad Juarez; Juarez University of the 
State of Durango; University of Guanajuato; Autonomous University of Guerrero; Autonomous 
University of the State of Hidalgo; University of Guadalajara; Autonomous Mexico State 
University; Michoacán University of San Nicolás de Hidalgo; Universidad Autónoma del 
Estado de Morelos; Autonomous University of Nayarit; Autonomous University of Nuevo 
León; Benito Juárez Autonomous University of Oaxaca; Meritorious Autonomous University 
of Puebla; Autonomous University of Queretaro; University of Quintana Roo; Autonomous 
University of San Luis Potosí; Autonomous University of Sinaloa; Technological Institute 
of Sonora; University of Sonora; Juarez Autonomous University of Tabasco; Autonomous 
University of Tamaulipas; Autonomous University of Tlaxcala; Veracruz University; Yucatan 
Autonomous University; and Autonomous University of Zacatecas. 


1.4.3 Institutional Coordination Structure 


According to the SEP, there are two levels of coordination in University, namely: The 
federal and the state. The first is characterized by the national scope of its policies and 
programs; and the state level is distinguished by being directly linked to the government of 
each of the 31 states and the Federal District. 


Now, in the sphere of the federal government there are two undersecretaries of State 
that have to do with University: The Undersecretary of University and Scientific Research 
(SESIC) and the Undersecretariat of Technological Education and Research (SEIT). The first 
coordinates and allocates resources to the UPs, and also administers the federal subsidies 
of the autonomous public universities, as well as that of the technological universities.º In 


8 Each of the states of the Republic has at least one autonomous university. Additionally, there are some autonomous 
public higher education institutions by decree of the federal executive branch. 

9 Among the SESIC's tasks, the following stand out: 1) Coordination of programs related to autonomous public uni- 
versities, technological universities and private universities and higher education institutions; the administration of the 
Comprehensive Program for Institutional Strengthening, PIFI, (whose first planning exercise took place in 2001), of 
which the Fund for the Modernization of Higher Education, FOMES, is a part; 2) The Investment Fund of State Public 
Universities with Evaluation of ANUIES, FIUPEA; and the Multiple Contributions Fund, FAM; the Support Program for 
University Development, PROADU; the Program for the Normalization of Administrative Information, PRONAD; and as 
part of the academic field, the Teacher Improvement Program, PROMEP; 3) The publication of national statistics on hi- 
gher education; 4) The promotion of higher education evaluation policies; 5) Issuance of the professional license; 6) Your 
link with ANUIES; 7) Keep a register of copyrights; 8) Be in charge of designing and executing the policy of the federal 
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addition, it does not have executive functions, nor does it participate in the preparation of 
curricula, but through the Comprehensive Program for Institutional Strengthening (PIFI) it 
stimulates the fulfillment of certain priorities; and, it has no interference in the government of 
the universities, nor does it influence the designation of its authorities (SEP, 2008).!º 


For its part, the SEIT coordinates federal policies for the technological institutes 
subsystem. This subsystem has the National Polytechnic Institute and 189 technological 
institutes, of which 23 were created during the beginning of this decade (Cf. Ibarra, 2002). In 
the Undersecretariat, the central authority has powers to: 1) Regulate teaching activities; 2) 
Design the curriculum; 3) Appoint the directors of the institutions and manage their budgets, 
and the labor relations of teachers and workers with the National Union of Education 
Workers (SNTE). The public system also corresponds to some professional schools such 
as the National School of Anthropology and History, the schools of the National Institute of 
Fine Arts, the agricultural University institutions, and the professional schools of the Armed 
Forces. 


In addition, it is necessary to point out that, in the field of state governments, there 
are different administrative units within the secretariats of public education responsible for 
University, which can be sub-secretaries, or general directorates of University. Likewise, it 
must be considered that in each of the States there is a State Commission for the Planning 
of University (COEPES), whose operation varies between entities. 


As regards the federal government, state governments and institutions have 
established policies and mechanisms for two decades in order to improve the SES. The 
current system, derived from the National System for Permanent Planning of University 
(SINAPPES), has registered moments of high effectiveness, but also some of reduced 
productivity. This is due, among other things, to the fact that the National Coordination for the 
Planning of University (CONPES) has not had a regular operation, and the state instances, 
fundamental for the development of HE in the states, continue to be unconsolidated and not 
have had a proper job. 


In addition, within the coordination of the SES, in Mexico there is the decisive 
participation of non-governmental organizations such as the National Association of 
Universities and Institutions of University (ANUIES), which has worked closely with the 
government in recent years around the design of University policies; even more recently, 
the National Center for Superior Evaluation (CENEVAL); and finally, the Council for the 
Accreditation of University (COPAES). 

Towards the end ofthe 1980s, a series of commissions made up of federal government 
officials and rectors or directors of the SES were formed, and as a result of such a strategy, 


government towards public universities (Vid. SEP, 2008). 

10 SESIC currently has among its special programs the PIFI (Integral Program for Institutional Strengthening), PROMEP 
(Program for the Improvement of Teachers), FOMES (Fund for the Modernization of Higher Education), PRONABES 
(Program National Scholarship for Higher Education), among others (Ídem). 
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the National Commission for the Evaluation of University (CONAEVA) emerged. Since then, 
the following evaluation schemes have been carried out, namely: 1) Self-evaluation by 
HEIs; 2) The inter-institutional evaluation entrusted to the Inter-institutional Committees for 
the Evaluation of University (CIEES); and 3) The evaluation of the system and subsystems 
in charge of the undersecretaries of the SEP and the ANUIES. Add, finally, that since 1993 
there has been CENEVAL, and since 2000 with COPAES, an instance endorsed by the SEP 
to issue official recognition to the accrediting bodies of the academic programs of HElIs in 
Mexico. 


1.4.4 University Institutions in Mexico 


In the legal framework of educational matters in Mexico, and specifically the General 
Education Law, public and private education are included within the national educational 
system; and also regulates in chapter V, the education imparted by individuals." 


In the country, SE has the public and private or private modality; both constitute 
the SES, which, as already noted, is made up of more than 1,500 institutions with different 
profiles and missions (SEP, 2001; 186). The public system comprises almost 70% of the 
total enrollment, although this proportion tends to vary between the levels of the system 
(Idem). 


In addition, educational institutions in Mexico have three options for their study 
programs to be officially recognized, namely: 1) From the autonomous institutions;!? 2) By 
incorporating;'* and 3) By granting recognition of official validity issued by the federal or 
state government. !! 


In recent times, the offer of educational programs in private or private University 
institutions (IESPP) has expanded significantly, thereby contributing to the professional 
training of Mexicans. Today, there are private institutions in all the states of the Republic. 
In addition, the government has contributed to improving the requirements and procedures 
for granting the Recognition of Official Validity (RVOE), which is granted by both federal 
and state governments and also by autonomous public universities, transferring their 
incorporation of studies to private institutions. 


Also, it is worth noting the body that represents private institutions is the Federation 
of Private Mexican Institutions of University, A.C. (FIMPES). 


11 Cfr. Articles 10º, 54º, 55º, 56º, 57º, 58º and 59º of the General Education Law. 

12 The respective organic law, issued by the federal or state legislature, previously grants official recognition to the study 
programs of the autonomous universities. In addition, autonomous universities have the power to grant official validity 
-through the figure of “incorporation”- to the educational programs of non-autonomous institutions. 

13 The autonomous universities assume the responsibility of ensuring that the programs offered by private institutions 
meet the curriculum they teach, and under the same standards in force of the same institution. 

14 This is an intermediate option for those already mentioned. They do not have the flexibility of autonomy, but neither 
do they have the rigidity of incorporation. Private institutions can offer their own curriculum and standards, as long as it 
has been authorized by the corresponding government agency. 
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Returning, according to data from the Directorate of University, in Mexico, University 
institutions (IES) are classified as public and private. Within the public ones are the 
universities, both state (UPE) and federal (UPF), both enjoy autonomy; there are also the 
so-called Solidarity Support State Public Universities (UPEAS); polytechnic universities 
and technological universities. Likewise, there are technological institutes, which, similar to 
universities, exist at the state and federal levels. On the other hand, there are the schools 
of the Army and the Navy, the normal schools, and others. And, for their part, private HEIs 
have universities, schools, institutes, centers and others. 


In the same vein, the ANUIES classifies the SES into six subsystems, namely:!8 
1) Public Universities (UP),!º 2) Technological Universities (UT),!” 3) Private or Private 
Universities (UPP),'* 4) Other Public Institutions (OIP),!º 5) Technological Institutions (IT),?º 
and 6) Normal Education (EN).?! There are also other typologies of HEls, such as the 
one approved by the ANUIES General Assembly in 1999, based on the 1997 UNESCO 
International Standard Classification of Education (ISCED), and in which what is related 
to the classification of programs and the establisnment of desirable minimums, in terms of 
the level and time of dedication of the academic staff. Based on these criteria, six types of 
them are proposed, namely:?2 1) IDUT: Institutions of University, focused predominantly on 
the transmission of knowledge, and that offer programs exclusively at the level of higher 
university technician or associate professional; 2) IDEL: Institutions of University, whose 
main activity is focused on the transmission of knowledge, and which offer programs 


15 Cfr. ANUIES, 2000. 

16 There are 45 institutions of this type, whose functions are teaching, research and extension of culture and services. 
This subsystem is represented by federal and state universities, with most of the autonomous public universities. In ad- 
dition, it attends approximately 50% of the country's research, 52% of undergraduate students and 48% of postgraduate 
students. 

17 They are decentralized public organisms of the state governments; and they are present at the federal, state and 
municipal levels, having been created since 1991. Their study programs last two years, training associate professionals. 
Thus, from 1998 to 1999 there were 36 technological universities in 19 entities, serving 1.1% of enrollment in higher 
education. 

18 There are 976 organizations, not including normal schools; and they are classified according to their official name in 
five groups: a) Universities: 306; b) Institutes: 256; c) Centers: 239; d) Schools: 88; and e) Other institutions: 87. Such 
universities require the RVOE, the SEP or the state governments, or they may be incorporated into a public educational 
institution authorized to do so. This subsystem attends to 27.6% of enrollment in undergraduate degrees and 36.5% in 
postgraduate studies. 

19 These are 67 institutions not included in the previous two; serve 1.1% of the undergraduate population, and 7.5% of 
the postgraduate. 

20 There are 147 institutions that represent 19% of undergraduate enrollment and 6% of postgraduate students. Of the 
previous number, 102 are coordinated by the SEP through the Center for Research and Advanced Studies of the IPN, the 
National Polytechnic Institute (IPN), the Federal and Agricultural Technological Institutes, those of Marine Sciences and 
a Forest Technological Institute. Of these, 45 are decentralized agencies of state governments. The offer of its programs 
lasts 3 years: 2 of the common core and 1 of specialization. Likewise, they favor a quick exit to the labor market and open 
the possibility of continuing with higher studies. 

21 This level prepares teachers in different types and levels of the National Education System. Their program lasts 4 to 
6 years. You have the possibility of obtaining a bachelor's degree in preschool, elementary education, secondary educa- 
tion, special education and physical education. Of all its schools, 220 are public and 137 private. They represent 11.8% 
of the population in higher education. 

22 Cfr. ANUIES, 1999. 
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exclusively or mainly at the undergraduate level; 3) IDLM: Institutions of University, whose 
main activity is focused on the transmission of knowledge, and which offer programs at 
the bachelor's and postgraduate level up to the master's level; 4) IDILM: Institutions of 
University oriented to the transmission, generation and application of knowledge, and that 
offer programs at the undergraduate and postgraduate level (predominantly at the master's 
level; eventually they have a doctorate program); 5) IDILD: Institutions of University oriented 
to the transmission, generation and application of knowledge, and that offer programs at 
the undergraduate and postgraduate level up to the doctorate level; 6) IIDP: Institutions of 
University, whose main activity is focused on the generation and application of knowledge, 
and which offer academic programs almost exclusively at the master's and doctoral level. 


1.4.5 The National Association of Colleges and Institutions of University 
(ANUIES) 


The HElIs of Mexico are faced with the imperative of facing the new demands 
and social needs related to the quality of the university system, after a stage that was 
characterized by the objectives of quantity and access, as well as the requirement to 
promote transparent evaluation systems. and quality assurance. For this reason, Mexican 
universities are currently concerned about increasing the quality of teaching, research and, 
in general, that of all the services it provides. 


The HEIs, therefore, have progressively adopted measures of change and adaptation 
to the new environment and, grouped around the National Association of Universities and 
Institutions of University (ANUIES), have established policies for their strengthening, within 
which they find, mainly, the institutional evaluation processes introduced by the National 
Commission for the Evaluation of University (CONAEVA), and external evaluations have 
also been registered, such as the one carried out at the request of the Ministry of Public 
Education (SEP) by the OECD in 1994. 


Since its creation in 1950, ANUIES has participated in the formulation of national 
programs, plans and policies, as well as in the creation of organizations oriented to 
the development of Mexican University. In addition, this non-governmental and plural 
Association brings together the main University institutions in the country to promote their 
comprehensive improvement in the fields of teaching, research and the extension of culture 


and services. 


The body is made up of universities and institutions of University, both public and 
private, from all over the country, which serve 80% of the enrollment of students who are 
pursuing undergraduate and postgraduate studies. In addition, “it promotes the development 
of University in the various regions and states of the country within the framework of 
federalism and the strengthening of the different institutional modalities. To this end, it 
designs and promotes educational initiatives, proposes and specifies State policies on 
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the matter, considering the attributions of civil society organizations and the spheres of 
competence of the three levels of government, and carries out strategic studies to foresee 
the main future trends. and support decision-making to consolidate the University system in 
Mexico. Articulates and represents the academic and administrative interests of its affiliates 
before the instances of the executive, legislative and judicial powers at the federal, state and 
municipal levels and before public and private, national and foreign organizations related to 
University” (ANUIES, 2007: 14). 


This group, likewise, is recognized by society and its institutions as the most 
representative non-governmental organization of the country's University system and the 
main non-governmental interlocutor in matters of University in Mexico, whose opinion is 
taken into account for the elaboration of state policies for its development and financing; Itis 
the association of University institutions most recognized by international organizations for 
the development of exchange projects and academic cooperation; Itis an incubator institution 
for strategic projects that generate changes and promote the sustainable development of 
University; carries out projects that articulate the efforts of affiliated institutions aimed at 
preserving, promoting and disseminating regional and national cultural wealth, within the 
framework of universal culture; plays an important role in strengthening University planning 
agencies in the states; actively participates in the establisnment of quality standards and in 
the processes of planning, evaluation and accreditation of University; and it has academic 
criteria by type of institution to enter and remain in the Association and supports affiliates in 
improving their performance (Idem). 


The collegiate bodies of the Association are instances of broad participation of 
the members of the affiliates and constitute effective spaces for decision-making and 
establishment of policies of collaboration and development of the University system. The 
complex reflects the heterogeneity of the system by responding to the various interests or 
needs of the institutions and subsystems, and also supports integration at the regional level 
in interests and academic needs of University institutions, while maintaining time a solid 


union at the national level. 


Finally, in orderto fully comply with its commitments, ANUIES has the following specific 
objectives: 1) Promote the integral and permanent improvement of the quality and coverage 
of the programs and services offered by the affiliated institutions; 2) carry out strategic 
studies on University to anticipate changes, design and coordinate policies, and support 
decision-making; 3) articulate the academic and administrative interests of the associated 
institutions and represent them before government agencies and public and private, national 
and foreign organizations; 4) promote complementarity, cooperation, internationalization 
and academic exchange of its members; 5) provide solutions to the problems of University 
and options for its development with quality at the national, regional and state levels; 6) 
promote inter-institutional projects and activities that foster the convergence of interests of 
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associated institutions at the national, regional and state levels; 7) promote the national, 
regional and state exchange of information, services and academic and/or specialized 
personnel among associated institutions for better communication and performance of 
common tasks; and, 8) promote relations and the establishment of agreements with national 
and foreign organizations, as well as with the social and productive sectors (Idem). 


21 THE EVALUATION OF THE QUALITY OF UNIVERSITY IN MEXICO 


2.1 Background and Establishment of Evaluation Policies of University in 
Mexico 

Before 1980, quality was not talked about much in the country, nor was the term 
evaluation of University included, although there were equivalent expressions. The 
development of University was due to a stage of unregulated quantitative expansion, in 
which growth followed a political rationality, subordinated to the interests of power groups 


within University institutions. 


In her article “The Mexican experience in institutional university evaluation in Mexico”, 
Llarena de Thierry (1997) introduces the context that fosters the evaluation mechanism 
in Mexico by identifying four fundamental elements for its implementation, namely: 1) 
Variables of the international context; 2) the socio-economic situation in Mexico; 3) the 
situation of Mexican University; and, 4) the planning-evaluation that was created in 1978 
with the National System of Permanent Planning of University. 


Therefore, in Mexico, the evaluation of HEIs constitutes a task that has been carried 
out for several years, and to the extent that higher quality is demanded in educational 
institutions, the need to evaluate the quality of HEIs has been generated. all the elements 
that comprise it. And it is that HEIs can be evaluated in their regulations, in their study and 
postgraduate plans and programs, in teaching activity, in their students, in infrastructure, 
and for each aspect to be evaluated there are programs with their respective instruments. 


In this line, various agencies were also created to carry out the evaluation schemes, 
under certain criteria and standards established for it. This way, in 1979, at the initiative of 
ANUIES, the National Coordination for University Planning (CONPES) was created, which 
was made up of the General Council of ANUIES, officials of the SEP and was chaired by 
the incumbent of the latter. This Coordination instituted the instances and processes of 
institutional, state and regional planning, and since then it has generated national policies 
regarding University. 

Thus, regarding the accreditation of curricular programs, the evaluation in 1978 
began with the start-up of CONPES in coordination with the SEP; and as far as planning is 
concerned, in 1979 the creation of the National System for Permanent Planning of University 
(SINAPPES) was finalized. 
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Until then, however, the evaluation was not standardized nor was it in charge of 
a specific body, so in 1989 the evaluation of University was institutionalized in Mexico 
with the Program for Educational Modernization 1989-1994 of the Federal Government. 
In this program, the permanent internal and external evaluations of the institutions were 
established as a priority action, to promote the improvement of the quality of the educational 
programs and services that were offered. In addition, the program defined the modernization 
of education as quality, efficiency, coverage and innovation, that is, a dynamic movement 
towards a qualitative change in educational processes. And he also pointed to evaluation as 
the fundamental strategy to achieve modernization (SEP, 1990: 27). Thus, the government 
made clear its intention to create an instance that would integrate and articulate a national 
process of evaluation of University. 


In order to make this objective a reality, in 1990, at the ANUIES Assembly, the 
establishment of a National University Evaluation System (SNEES) was proposed. CONPES 
participates and works collegially with government and university authorities designing the 
national strategy for the creation and operation of the SNEES: The National Commission for 
the Evaluation of University (CONAEVA). 


This body included among its purposes those related to promoting a national 
evaluation process through the formulation of criteria and general guidelines, as well as 
proposing policies and actions aimed at improving the current conditions of University. 
Thus, CONAEVA was based on three lines of action: Institutional evaluation (self- 
evaluation), institutional evaluation by the Interinstitutional Committees for the Evaluation 
of University (CIEES), and interinstitutional evaluation of academic programs and functions 
of the institutions, through the evaluation mechanism of qualified peers of the academic 
community, that is, the global evaluation of the University system and subsystems, carried 
out by the Undersecretary of University and Scientific Research (SESIC), the Undersecretary 
of Education and Technological Research (SEIT), the Council of the National Technological 
Education System (CONSNET) and ANUIES. 


In general terms, the approach focused on the analysis and assessment of specific 
undergraduate or postgraduate programs, rather than institutions. Subsequently, other 
evaluation mechanisms associated with postgraduate studies were established through 
the National Council of Science and Technology (CONACYT), which implemented annual 
national competitions for the integration of the list of excellent postgraduate courses, through 


external academic peer evaluation mechanisms. 


However, given that these instances were not sufficient to promote the evaluation 
process, it was imperative to create others such as the Interinstitutional Committees for 
the Evaluation of University (CIEES) to evaluate specific programs by areas of knowledge. 
Thus, in 1991, CONPES established the CIEES as non-governmental organizations made 
up of academic peers from different University institutions in the country. And the main 
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functions assigned to the CIEES were: a) The diagnostic evaluation of academic programs 
and institutional functions in a given area and the accreditation and recognition of academic 
units or specific programs, to the extent that they accredit quality standards, conventionally; 
b) the ruling on projects or programs that request additional financial support, at the request 
of the public administration agencies that provide these resources; c) advice, at the request 
of the institutions, for the formulation of programs and projects and for their implementation 
(CIEES, 2009). 


In 1990, within the ANUIES, at a national meeting, the SEP announced its intention 
to require the evaluation of the institutions. However, the resistance reaction that was 
attempted under the protection of university autonomy was immediate but could not be 
sustained for long because the SEP conditioned the allocation of the subsidy budget to 
the prior performance of self-assessments. The launch of the evaluation process was 
irreversible. 


From 1991 to 2006, the evaluation of educational programs and institutional 
functions has been carried out; and in 17 entities of the country, 100% of almost 3,000 
academic programs and institutional functions have already been evaluated, with evaluation 
methodologies and frameworks that have a wide repertoire of categories and components, 
in which international criteria and standards have been retaken. Up to February 2008, 
the CIEES have delivered 2,609 evaluation reports and 961 are in the process of being 
prepared. 


In addition, in 1993, the National Coordination for the Planning of University 
(CONPES) and the National Council of ANUIES jointly proposed to create an institution 
that would be responsible for a new line of evaluation, the evaluation of results. Thus, at the 
beginning of 1994, the National Evaluation Center for University, A.C. (CENEVAL), a non- 
governmental and self-financing body, was launched, which would carry out the necessary 
actions to carry out the exams (EXANI-II and EGEL)., as well as the National Entrance 
Examination for Upper Secondary Education (EXANI-I). 


In order to influence the problem of the accreditation of academic programs, the 
General Assembly of ANUIES agreed in 1997 to promote the creation, by CONPES, of a 
non-governmental organization whose purpose was to regulate the accreditation processes, 
and which would provide certainty of the technical and operational capacity of specialized 
organizations dedicated to the accreditation of academic programs for the improvement 
of the quality of University. After carrying out an extensive process of analysis to define its 
structure, composition and functions, in October 2000 the Council for the Accreditation of 
University, A.C. (COPAES) was formally instituted, which is authorized by the SEP to confer, 
After a rigorous evaluation process, the official recognition of those accrediting bodies that 
have demonstrated their effectiveness, efficiency, and reliability in accrediting academic 
programs at the Bachelor's, Higher University Technician, or Associate Professional level, 


Evaluation of the Quality of University in Mexico 5 


in defined areas of knowledge, in the public and private institutions throughout the country.?º 


Before concluding this section, it is worth considering the most relevant premises 
regarding evaluation as a strategy for the modernization of University in Mexico (SEP, 1990: 
35): 1) Evaluation must not be an end in itself itself, but acquires its meaning to the extent 
that the development of this educational level is supported; 2) education must be an integral 
part of the planning processes for academic and support tasks, and not an overlapping 
process to comply with administrative requirements; 3) the evaluation must be understood 
as a permanent process that allows to improve, gradually, the academic quality, and not 
as a cut from which a complete and objective knowledge of the situation of University can 
be expected. Therefore, it must incorporate a diachronic vision (over time), which allows 
evaluating progress and achievements, identifying obstacles and promoting actions for 
academic improvement; and, 4) the evaluation processes that were promoted must have an 
impact on plans and programs for the development of education in different spheres, from 
the institutional to the national. 


From the formation of the new global geopolitical and geoeconomic blocks, an 
international trend of evaluation, accreditation and certification of University institutions 
(IES) has been generated in two aspects: One associated with the Free Trade Agreement 
and the other with the European Union. European. The first, led by Canada, promotes the 
validation of quality criteria and indicators for University; the second, associated with the 
Alfa Project, is called the University Network for Quality Assessment. This, therefore, raises 
the imperative tone of evaluation in HEls in the country. 


Finally, in the period 1994-2009, Mexico faced two new economic crises (1994-1996 
and 2007-2009). These events definitely reaffirmed the State's policy of evaluating the 
performance of HEls in order to grant economic incentives to academics and additional 
financing for IPES. Thus, during the presidencies of Zedillo (1994-2000), Fox (2000-2006), 
and Calderón (2006-2012), diagnostic and accreditation organizations (CENEVAL, CIEES, 
COPAES, and CONACYT) were strengthened. In addition, the recommendations of the 
ANUIES have been considered, and the National Institute for the Evaluation of Education 
(INEE) was created by decree, which, like the SEP, works under the tutelage of the 
Presidency of the Republic. 


2.2 Usefulness of the Quality Assessment Process for Institutional 
Development in Mexico 


According to Álvarez and Topete: “It has been observed that for an evaluation process 


23 Any national or foreign organization can accredit academic programs (careers and postgraduate degrees), but only 
those at the bachelor's degree and university or associate professional level that receive recognition ffom COPAES have 
the authorization of the Ministry of Public Education. CONACYT also evaluates postgraduate academic programs for 
inclusion in the National Postgraduate Quality Program (PNPC) -formerly the National Postgraduate Register (PNP)-. 
The latter has specific standards to evaluate postgraduate degrees, master's degrees, doctorates and specialties in the 


health area, but only those of 2 years or more. 
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to be useful and serve institutional development, it must meet the following conditions: 1. 
Create an evaluation culture so that the processes are true and authentic. 2. That the results 
of the evaluation be used for institutional development. 3. That the subjects know the results 
of the evaluation and who is evaluating them. 4. That the evaluation take into account 
the conditions of the educational production processes. 5. That the evaluation process 
be continuous and permanent, and have the representation of the various academic and 
administrative bodies involved. 6. That the evaluation is not confused with the filling of 
forms and that the generating instances of the information guarantee the reliability and 
validity of the information. 7. That there are monitoring mechanisms in the implementation 
of the quality plan. There is no improvement plan without follow-up. 8. That the statistical 
data for the evaluation be an authentic support and the numbers clarify the problem, and 
not the other way around. 9. That the management units regarding educational quality be 
formalized, so that they contribute to a culture of positive and non-reactive evaluation in 
the face of the evaluation processes. 10. Legitimate evaluation can be a valuable means of 
interaction between organizational areas” (Álvarez and Topete, 1997). 


2.3 Formatis of Institutional Evaluation in Mexico 


In the specific case of Mexico, three types or formats of institutional evaluation can 
be distinguished, namely: 


1.) The internal, institutional or self-assessment, by the HEIs themselves; 


2.) The inter-institutional evaluation of services, programs and projects in the various 
functions and areas of University through the peer evaluation mechanism, that is, the 
external evaluation of educational programs by committees made up of academic 


peers; Y, 


3.) The evaluation of the subsystems and the University system by specialists. 


In the following sections, a succinct characterization of the institutional evaluation 
formats in Mexico indicated here will be made. 


2.8.1. Internal, Institutional or Self-assessment 


The institutional evaluation carried out by the HEIs themselves (SEP, 1990: 53 and 
54) aims to achieve an evaluative analysis of the organization, operation and results of 
the academic and administrative processes that they develop. In order to facilitate the HEI 
evaluation process, CONAEVA formulated and proposed a series of analysis categories, 
criteria, indicators, and standards that serve as a guide for institutions to carry out their self- 
evaluation. And likewise, he made the proposal of the basic information they must have to 
carry it out. 
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The evaluation (and accreditation) process often comprises three stages: self- 
assessment (self-study), external evaluation, and the final report (preparation, discussion, 
and dissemination). 


Self-assessment entails a fundamental ethical attitude. It is normally carried out by 
committees of specialists appointed by the academic unit itself, who carry out their task 
through guidelines or protocols previously defined by teams of specialists. These committees 
coordinate the collection of information, the development of performance indicators, their 
analysis and the preparation of a self-assessment report (or internal evaluation). 


In the external evaluation, for its part, a team of specialists from outside the evaluated 
academic unit reviews the self-assessment report. It starts with a visit to the academic unit 
that culminates with the presentation of a final report. This is generally discussed by an 
independent body, whose decisions influence the allocation of resources, the academic 
recognition of the evaluated units, or the accreditation of the evaluated programs. 


The importance of carrying out self-assessment processes is manifested in the 
policies recommended by UNESCO and by the OECD. The first points out that the evaluation 
acquires a broader meaning than just assessing the educational offer and teaching methods, 
since it must also consider financing, management, general orientation and the achievement 
of long-term objectives, which will allow providing students with different agents a better 
knowledge of their action, spread the capacity for innovation and publicize successful 
initiatives and their conditions of realization (UNESCO, 1996: 177 and 178). For its part, 
the OECD, in the document “Review of Superior Evaluation Policies”, recommends that the 
evaluation of institutions be carried out through the normality of self-evaluation based on 
its own indicators that can be grouped into four categories that can be evaluated., namely: 
Students, study plans and programs, personnel and financial resources (OECD, 1997: 54). 


2.3.2 External or Interinstitutional Evaluation 


The inter-institutional evaluation is part of two simultaneous evaluation processes: 
The institutional evaluation, in charge of the institutions themselves, and the evaluation of 
the University system, in charge of the SEP and the ANUIES. 


The inter-institutional evaluation is aimed at knowing and assessing the operating 
conditions and the quality of the services, programs and projects, as well as the processes 
and results of the institutional work. The functions of inter-institutional evaluation are: 
Diagnostic evaluation in a specific area of knowledge, accreditation and recognition of 
specific programs. Opinions on projects or programs that solve economic support and 
advice to institutions for the formulation of programs and projects. 


In order to carry outthe inter-institutional evaluation, the Inter-institutional Committees 
for the Evaluation of University (CIIES) were created, located in CONAEVA under the 


Evaluation of the Quality of University in Mexico ER 


auspices of the Undersecretary of University and ANUIES, also called peer committees. 
Such committees are made up of academics of recognized prestige in the area of their 
specialty, appointed based on the proposals of the educational institutions; and they have 
a double character, since they belong to their national academic institutions and participate 
in the committees. 


Currently, nine evaluation committees have been installed, in the following areas of 
knowledge: Natural and exact sciences; agricultural sciences; engineering and technology; 
Health Sciences; social and administrative sciences; education and humanities; architecture, 
as well as the committees that evaluate the administrative and dissemination and extension 
functions (CIEES, 1994). 


Each committee has established its own criteria, indicators, norms or minimum 
standards and its particular way of carrying out the evaluation of the programs or of the 
function or of the institutions. The development of their reference frameworks and the 
definition of their own internal evaluation strategies have been one of the main internal 


tasks. 


In this first stage, reference is made to those institutions that request it. Before the 
visit, a pre-diagnosis is made, product of the information provided by the IES and the analysis 
of their own self-assessments. The visit to the institution, as well as the interview with 
professors, students, researchers, authorities and productive sectors of their environment 
with fundamental elements that allow determining the degree to which the norms, profiles 
and minimum quality standards previously established by the institutions themselves 
are met. institutions. At the end of the visit, the Committee prepares a report and issues 
the necessary recommendations to correct or solve problems that it has detected. The 
recommendations of the Committee are of an indicative nature and their implementation 


depends on each program or institution. 


The evaluated institution is requested to analyze the pertinence of each one of the 
recommendations and send its comments to the Committee, in order to continue the internal 
learning process for the evaluation. Due to the importance of the information contained in 
the reporis, they are handled confidentially. The report is delivered in the first instance to 
the rector or director of the institution, and is made known to CONPES. These are the main 
users of the results of the interagency evaluation. In their time of operation, the CIEES have 
organized their own work and defined their frames of reference, as well as having evaluated 
almost all the HEIs in Mexico in terms of their administration and management. Each 
committee also has its frame of reference and includes in its evaluations the fundamental 
characteristics of the evaluation. It is up to each institution to define which recommendations 
apply to their academic projects and how they apply them. 


Currently, the CIEES seek to collaborate closely with professional associations, 
national associations of schools and faculties, and with organizations that carry out activities 
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related to theirs to establish special instances that address the program accreditation 
work originally entrusted to the CIEES. be carried out by collegiate bodies in which the 
committees participate, together with professional associations, school associations, the 
employer sector and other interested entities. And, at first, the following councils have 
been constituted for the sake of teaching accreditation: Engineering (CACEI), Veterinary, 
Accounting and some branches of Medicine; and its form of organization and collegiate 
participation is being used as a model to set up similar councils in other areas of University. 


2.3.3 Accreditation 


However, another procedure that allows evaluating the institutional level is 
accreditation. This is a mechanism to determine the global level of a University institution, 
a certification of the academic quality of an academic program or unit, or a judgment on the 
consistency between the objectives, resources and management of an academic unit. The 
accreditation process usually includes the self-assessment, the external assessment, and 
the final report. 


Similarly, accreditation can be institutional or academic. The first examines the 
overall characteristics and services of the institution. The accreditation of an institution 
means that it has been evaluated and verified that it has well-defined objectives, appropriate 
resources, an installed and stable capacity to offer educational services of a certain quality, 
and that it meets minimum operating criteria (standards). This process does not imply that 
all programs or academic units are of the same quality, nor that any particular program is 
accredited. It is generally voluntary, as in the United States (Council of Post-Secondary 
Accreditation, COPA; or the regional associations of University institutions, for example the 
Southern Association of Colleges and Schools, SACS). 


In Mexico, accreditation is being implemented through the Committees who, at 
the end of 1999, designed mechanisms that would allow for a framework of reference for 
accreditation. It is necessary to point out that the processes of institutional evaluation and 
accreditation have been questioned since they intrinsically contain a dimension referring to 
the exercise of power (internal-external), control and learning. It can be designed to inform a 
third party or to inform the agents involved, a means of technification and control or a means 
of self-learning (of clarification and emancipation). In this sense, participation builds trust 
in the evaluation process, and it is also necessary to take care of the choice of evaluators, 


whether internal or external. 


Finally, regarding the above, Angulo points out that “an educational project includes 
the political principles that make explicit the type of school, the forms of socio-educational 
relationship and the teaching-learning and organizational dynamics that are to be carried 
out. For this reason, this project, which requires generalized and public discussion of all 
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those directly involved in the work of the centers, can become a platform for participation 
and knowledge of civil society, teachers and students in coordination and with the support 
and advice from other instances” (Angulo, 1992: 23). 


2.4 The Assessing and Accrediting Instances and Organizations of University 
in Mexico 

In the document called University in the 21st century (ANUIES, 2004), a program 
regarding the evaluation and accreditation of University in Mexico was proposed. According 
to the diagnosis made, insufficiencies were identified: the results of the evaluations are 
poorly articulated with decision-making and have little roots in the academic communities; 
the evaluation of academies has privileged individual production over collective work, the 
disconnection between the various evaluation, accreditation and certification processes and 
the lack of a national agreement in this field. 


Thus, ANUIES proposed to promote this policy to “consolidate the national system 
of evaluation and accreditation of University, with the participation of all the actors involved, 
which strengthens and articulates the different organizations with responsibilities in terms 
of evaluation, accreditation and certification, and with the ultimate purpose of influencing 
quality improvement and assurance” (Ibid., 186). Therefore, ANUIES ruled that these tasks 
must fall to independent and intermediate bodies between University institutions and public 
powers, and not directly to the State. 


From the perspective of ANUIES, the consolidation of a national evaluation and 
accreditation system would allow the achievement of three objectives: contribute to the 
improvement and quality assurance of programs and services offered by universities, 
recognizing and promoting institutional diversity; guarantee users of educational services 
that accredited academic programs meet quality requirements and have the infrastructure 
and suitable mechanisms and instruments to ensure the completion of their projects, and 
establish channels of communication and interaction between the various sectors of civil 
society in search of an education of higher quality and relevance. With a horizon of six years, 
the ANUIES raised a set of goals in terms of evaluation and accreditation. Among them they 
highlighted: The constitution of a council for the evaluation and accreditation of University, as 
a civil association, whose functions would be the normalization and official recognition of the 
organizations that carry out activities of external evaluation or accreditation of educational 
programs; the creation of specialized accreditation bodies for academic programs and their 
recognition by this council; promoting the updating of federal legislation on professions; the 
formation of a national register of good quality programs and the improvement of the criteria 
for granting recognition of the official validity of studies to private University institutions 
based on external evaluation and accreditation mechanisms. 


Likewise, ANUIES also requested the federal and state governments to establish 
complementary policies and programs to support universities, which consider the results of 
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these processes, to guarantee the balanced and sustained development of the University 
system. With regard to the federal government's policy, the National Education Program 
2001-2006 (PRONAE) designed a policy for the development of University with three 
strategic objectives: expansion of coverage with equity; University of good quality and 
integration, coordination and management of the University system. Regarding evaluation 
and accreditation was included in the second strategic objective. PRONAE established 
different lines of action for the improvement and assurance of quality, having as a focus the 
educational programs taught by the institutions. 


Thus, it was proposed, among other actions, to promote the diagnostic evaluation 
carried out by the CIEES; encourage the accreditation of programs by specialized 
organizations formally recognized by COPAES; encourage the formation of specialized 
accreditation bodies for educational programs of a non-governmental nature; establish 
criteria and procedures for the creation of the National Postgraduate Register (PNP) -SEP- 
CONACYT-; promote international recognition of the educational program accreditation 
scheme; formulate criteria for the evaluation and accreditation of academic programs that 
are taught through non-school and mixed modalities; grant extraordinary economic support 
to public institutions to ensure the quality of educational programs that have been accredited 
by organizations recognized by COPAES, or that are part of the PNP; promote the external 
evaluation of students upon admission and graduation for diagnostic purposes; improve the 
requirements and procedures for granting the Official Validity of Studies Registry (RVOE) 
of the superior type and strengthen the technical capacity of the federal government for the 
analysis of applications, as well as for supervision. 


2.4.1 The National Evaluation Council for Education Superior, A.C. 
(CENEVAL) 


CENEVAL was legally established in 1994 as a civil association,” which has 
allowed it to act autonomously as a professional organization, with sufficient closeness and 
independence from both the government, professional associations, educational institutions 
and companies.?º In addition, it was established as a labor competency certifying body, thus 


24 The members of the general assembly of associates of CENEVAL, A.C. (2004) are associations and educational 
institutions: National Association of Universities and Institutions of Higher Education, A.C. (ANUIES); Federation of Pri- 
vate Mexican Institutions of Higher Education, A.C. (FIMPES); National Polytechnic Institute (IPN); Monterrey Institute 
of Technology and Higher Education (ITESM); Autonomous University of the State of Mexico (UAEM); Autonomous 
University of San Luis Potosí (UASLP); Autonomous University of Yucatan (UADY); Popular Autonomous University of 
the State of Puebla (UPAEP); Technological University of Mexico (UNITEC). Associations and professional associations: 
Barra Mexicana Colegio de Abogados, A.C.; National College of Actuaries, A.C. (ANC); National College of Psycholo- 
gists, A.C. (CNP); Federation of Medical Colleges and Associations; Veterinarians and Zootechnicians of Mexico, A.C.; 
Mexican Institute of Public Accountants, A.C. (IMCP). Social and productive organizations: Academy of Engineering, 
A.C.; Mexican Academy of Sciences, AC; National Academy of Medicine, A.C.; ICA Foundation, A.C. and the Ministry of 
Public Education. (See CENEVAL Council, 2004: 7). 

25 Therefore, the CENEVAL has no obligation to be called to account both for legal responsibilities (honesty) or for its 


performance. 
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helping to identify and reassess the nation's human resources. 


In 1994, the CENEVAL applied 265,552 examinations, a figure that has increased, 
in 20083, to 2,689,104 (CENEVAL, 2000). The Center develops, mainly, two types of exams: 
the National Entrance Examinations (EXANI), which evaluate the fundamental skills and 
competencies, as well as the essential knowledge that those who aspire to continue their 
studies in upper secondary and University must have, as well as the General Examinations 
for the Graduation of the Degree (EGEL), which evaluate the essential knowledge and 
information that a recent graduate of the bachelor's studies must show. The CENEVAL has 
Technical Councils that function as collegiate bodies, they are in charge of the design and 
evaluation of the exams that the CENEVAL offers to individuals and institutions of higher 
and secondary education. 


In addition, the CENEVAL develops special programs for the accreditation of high 
school and certain degrees by people who acquired the necessary knowledge in a self- 
taught way or through work experience, based on Agreement 286 of the SEP; exams for the 
evaluation of professional skills, teaching practice, preparation for teaching and professional 
profile, and processes for the certification of job skills in accordance with the provisions of 
the National Council for Standardization and Certification (KNOW). 


Finally, the exams applied by this body are listed: 1) National Entrance Examinations 
for upper secondary, higher and postgraduate education (EXANI); 2) General Examinations 
for the Graduation of the Degree (EGEL) in administration, accounting, agronomic 
sciences, pharmaceutical sciences, international trade-business, law, nursing, industrial 
engineering, civil engineering, electrical engineering, electronic engineering, mechanical 
engineering, computer engineering, computing, chemical engineering, marketing, general 
medicine, veterinary medicine and zootechnics, dentistry, pedagogy — educational 
sciences — psychology, chemistry and tourism; 3) General Examination for the Graduate of 
Technical Nursing Professionals; 4) General Examinations for the Graduate of the Higher 
University Technician; 5) CONALEP exams; 6) Labor Skills Certification Exams; 7) National 
Pedagogical University exams; 8) Examinations of the National Public Security System; 
9) Examinations of Agreement 286 (SEP), Agreement 286 was issued by the SEP and 
establishes the guidelines applicable to the revalidation of studies carried out abroad and 
the equivalence of studies, or even to those who did not carry out formal studies but have 
acquired the knowledge corresponding to an area of knowledge through work practice. 


2.42 The Interinstitutional Committees for the Evaluation of Universities 
(CIEES) 


As noted in advance, in 1991 the Inter-institutional Committees for the Evaluation of 
University (CIEES) were created, the result of a collaboration agreement between University 
institutions and the federal government, within the National Coordination for the Evaluation 
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of University. Planning of University (CONPES) nine committees were integrated.? 


The CIEES identified five components of the evaluation that would be practiced: 
Comprehensive, objective, contextual, analytical, constructive and transcendent, and each 
committee took on the task of formulating its framework of reference for the evaluation 
(CIEES, 2008). 


According to information provided by the CIEES, up to February 2003, 17 states 
had evaluated 100% of the UP's academic programs. Over twelve years, 2,609 evaluation 
reports had been delivered and almost half a thousand were in the process of being 
prepared. However, the universe of evaluation has been profoundly unbalanced, being made 
up almost entirely (95%) by the PUs, which is indicative of the minimal participation of other 
types of institutions, such as private universities, which participate with 3%, technological 
institutes, with 1%, and technological universities, with the remaining 1% (CIEES, 2004). 


Likewise, it must be noted that the main product of the evaluation of the committees 
is reflected in a report with recommendations to the universities and academic entities that 
teach the evaluated programs, and that it is not made public. Altogether, in twelve years 
more than 40 thousand recommendations were issued. More recently, the CIEES have 
been carrying out their follow-up, and they report that a high proportion of the universities 
have complied with the recommendations received as a result of the evaluations, and 
that only a very small proportion have been dissatisfied with the evaluation reports. Other 
activities carried out by the CIEES have been promoting the training of academics in the 
field of evaluation; support for self-assessment and evaluation of academic programs at the 
request of various universities; support to various organizations and authorities; the opinion 
of projects and programs of the universities to be supported by the SESIC programs, and the 
impulse to the constitution of the accrediting organisms of educational programs (CIEES, 
20083). 


The evaluation of programs carried out by the CIEES, as well as that carried out 
for accreditation purposes by the accrediting bodies, is voluntary. As noted, there is no law 
that obliges University institutions to be externally evaluated and even less to attend to the 
recommendations that emerge from the evaluations. However, within the framework of the 
policies promoted by the SESIC starting in 2001, its authorities asked the CIEES to classify 
the programs evaluated into three levels, according to their academic consolidation and 
their proximity to accreditation. This way, the CIEES integrated a list of programs evaluated 
in relation to accreditation. 

In the first classification exercise carried out by the CIEES, they placed, among 1,288 
programs then evaluated, 473 programs at level 1 (level closest to accreditation), 578 at 


26 Architecture, Design and Urbanism Committee; Agricultural Sciences Committee; Natural and Exact Sciences Com- 
mittee; Health Sciences Committee; Social and Administrative Sciences Committee; Education and Humanities Commit- 
tee; Engineering and Technology Committee; Administration and Institutional Management Committee; and the Diffusion 
and Extension of Culture Committee. (Vid. CIEES, 2004: 5). 
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level 2 and 273 at level 3. By the end of 2002 reported, out of a total of 2,071 programs 
evaluated, 720 at level 1, 883 at level 2 and 468 at level 3. This means, according to the 
assessment of the CIEES, that a career is taking place ascendant in the improvement of the 
quality of the programs (SESIC, 2003). 


Although the evaluation results, and the corresponding classification of the CIEES, 
are not directly associated with the granting of resources, the fact that the universities 
participate in external evaluation and accreditation processes is. This was made clear in 
the guidelines for updating, formulating, and presenting the Comprehensive Institutional 
Strengthening Program (PIFI), which indicated attention to the recommendations of the 
CIEES as one of the inputs for planning, as well as verification of the level granted by the 
CIEES to the educational programs evaluated. 


Among the objectives of the PIFIs, two were related to the topic dealt with here: 
Improve the quality of the educational programs offered by the institution and achieve their 
accreditation by specialized organizations recognized by COPAES or temporarily classified 
at level 1 by the CIEES., as well as maintain the accreditation of those programs that have 
achieved it. The guidelines for updating the PIFI propose that universities analyze the 
evolution of the quality of educational programs, for which questions such as the following 
were raised: Which educational programs were evaluated for the first time by the CIEES in 
the course of the last two years?; what are the relevant recommendations?; how were they 
classified (1,2,3)? o Which of the CIEES level 3 educational programs managed to move 
to level 2 or 1 in the last two years? What did the improvement actions consist of? Which 
educational programs achieved their accreditation by organizations recognized by COPAES 
in the last two years? (Ibid., 91). 


Lastly, the CIEES have established university profiles based on four indicators: 
openness to external evaluation, consolidation, follow-up of recommendations, and program 
accreditation. They argue that in most PUs, in mid-2008, there is a culture of evaluation that 
is progressively rooted in the academic community and that there is a growing willingness 
to be evaluated externally. 

2.4.3 The Council for Education Accreditation Superior, A.C. (COPAES) 


ANUIES had been discussing different proposals to form an organization that would 
coordinate the different accreditation processes. In 1997, its General Assembly agreed that 
the associated institutions would promote the creation of a civil association, self-financing 
in the medium term, as a collegiate, plural, representative and autonomous entity, endowed 
with extensive academic and technical capacity, to articulate the organizations with functions 
of evaluation and accreditation. 


After three years of this ANUIES agreement, atthe end of the year 2000 the Council for 
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the Accreditation of University (COPAES) was formally constituted.),” as a civil association 
that restricted its action to the field of accreditation, unlike the ANUIES agreement, which 
proposed a body that would also have coordination functions in terms of evaluation, after 
the disappearance of the National Evaluation Commission that had been formed in 1990. 


COPAES was created as an association made up of legal entities, and despite being 
a non-governmental organization, it was constituted as “the entity trained and recognized 
by the Federal Government, through the Ministry of Public Education (SEP), to confer 
recognition in favor of organizations whose purpose is to accredit University academic 
programs offered by public and private institutions, after assessing their organizational, 
technical and operational capacity, their evaluation frameworks for the accreditation of 
academic programs, the administration of their procedures and its impartiality” (COPAES, 
2004:a). For this, the SEP signed an agreement whereby this agency of the federal 
government only recognizes programs accredited by organizations recognized by this 
council. 


COPAES has a regulatory function of the organisms that accredit educational 
programs; therefore, it does not directly accredit programs, but instead gives its endorsement 
to those programs accredited by a formally recognized specialized body. It seeks to avoid 
conflicts of interest and inform society and users of educational services about the quality 
of a higher level study program. To fulfill its function, the Council developed the General 
Framework for Accreditation Processes for Higher Level Academic Programs (COPAES, 
2004b), and in 2001 issued a call for organizations seeking recognition as accreditation 
bodies to submit to Evaluation. As of that year, various associations have submitted their 
application and by 2004 fifteen of them are recognized.? 


COPAES identifies different characteristics for all academic program accreditation 
processes: voluntary; integral; objective, fair and transparent; external, ethical and 
responsible; temporary (five years); reliable, and product of the work of people of recognized 


27 COPAES is an Association that will be integrated at all times by legal entities, public or private, of Mexican nationality. 
The constituent partners were: Ministry of Public Education (SEP); National Association of Universities and Institutions 
of Higher Education of the Mexican Republic, A. C. (ANUIES); Federation of Private Mexican Institutions of Higher Edu- 
cation, A. C. (FIMPES); Federation of Colleges and Associations of Veterinary Zootechnicians of Mexico, A.C.; College 
of Civil Engineers of Mexico, A.C.; Mexican Institute of Public Accountants, A.C.; Mexican Bar, Bar Association, A.C.; 
Mexican Academy of Sciences, A.C.; National Academy of Medicine of Mexico, A.C.; National Academy of Engineering, 
A. C. (Cfr. COPAES, 2004). 

28 Mexican Council for Accreditation of Design Programs, A.C. (COMAPROD); National Council for Teaching and Pro- 
fessional Practice of Chemical Sciences, A.C. (CONAECQ); National Council for the Quality of Tourism Education, A.C. 
(CONAET); Association for Accreditation and Certification of Social Sciences, A. C. (ACCECISO); Mexican Council for 
Nursing Accreditation and Certification, A. C. (COMACE); Accreditation Council for Teaching in Accounting and Admi- 
nistration, A.C (CACECA); National Council of Dental Education, A.C. (CONAEDO); National Council for Informatics 
and Computing Accreditation (CONAIC); National Association of Professionals of the Sea, A.C. (ANPROMAR); Mexican 
Committee for Accreditation of Agricultural Education, A.C. (COMEAA); Mexican Council for Accreditation of the Teaching 
of Architecture, A.C. (COMEAA); National Council for Teaching and Research in Psychology, A.C. (CNEIP); Engineering 
Teaching Accreditation Council, A.C. (CACEI); National Council of Education of Veterinary Medicine and Zootechnics, 
A.C. (CONEVET); and the Mexican Council for the Accreditation of Medical Education, A.C. (COMAEM). (Cfr. Ídem). 
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competence in the matter.?? The accreditation process is the one usually followed at 
the international level: Each accrediting body establishes an evaluation methodology, 
a reference framework, indicators and parameters for accreditation. Although these are 
specific to each one of the agencies, and can be exercised autonomously, they must adhere 
to the general provisions established by COPAES, as a regulatory agency. 


2.4.4 IThe National Council for Science and Technology (CONACYT) 
and the National Quality Postgraduate Program (PNPC) 


In 1991, the National Postgraduate Strengthening Program (PFPN) was formulated, 
with two aspects: The first, which had the character of accreditation, referred to the 
recognition of what is called good quality of postgraduate programs (CONACYT, 2003: 1), 
that is, they have consolidated academic bodies, high graduation rates and the necessary 
infrastructure for their proper functioning; and the second aspect was to support the 
continuous improvement of the quality of the programs, for which the educational institutions 
formulated their Comprehensive Programs for the Strengthening of Postgraduate Studies, 
PIFOP (ibid., 1-3), in a similar way to the PIFI. 


In 2002, the SEP-CONACYT National Postgraduate Register (PNP) was formed, 
which gradually replaced the previous Register of excellence programs. The new register 
included not only scientifically oriented programs, but also professionally oriented 
programs.*º Today, the PNP was replaced by the National Postgraduate Quality Program, 
PNPC, also under the command of SEP-CONACYT. 


The evaluation carried out by the SEP and CONACyT on postgraduate programs has 
had the purpose of granting financial resources from the federal government. The PNP and 
the PNPC have constituted a tacit form of accreditation of the quality of the programs. To be 
part of them, the evaluated programs have had to comply with the established parameters, 
which makes this process analogous to that carried out by the accrediting bodies of 
undergraduate programs. The difference is that for the postgraduate degree, accreditation 
is carried out by a public body, unlike the one carried out by the first ones, which are civil 
associations, despite the fact that external academic peers from educational institutions 
participate in the evaluation process. 


29 Cfr. Ibíd., 4y 5. 

30 As a result of the first call held that year, 204 educational programs from 34 higher education institutions were in- 
cluded in the register, of which 174 were classified as high level and 30 as internationally competent, according to the 
criteria used for the conformation of the register (CONACYT, 2003). 
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SYNTHESIS AROUND THE EVALUATION OF THE QUALITY OF 
EDUCATION IN THE PUBLIC UNIVERSITIES OF MEXICO: CR CRITICISM 
AND PROPOSAL 


11 CRITICAL VISION OF THE EVALUATION OF THE QUALITY OF EDUCATION 
IN THE PUBLIC UNIVERSITIES OF MEXICO 

As it has been pointed out in this work, today Mexico has different organizations 
and processes related to evaluation to ensure the quality of education in its PUs. However, 
after almost two decades of having several of these instruments, it becomes imperative to 
question whether they have been and are suitable for the proposed purposes, given the 
articulation they have with each other. 


Specifically, the CIEES and COPAES have the same universe, with the difference 
that the evaluation of the former is of a diagnostic nature and that of the latter is for 
accreditation purposes. The CIEES see themselves as peers who make a formative, not 
punitive, evaluation and make recommendations for improvement. On the other hand, the 
evaluation for accreditation is valued as a qualification that will be assigned to the program: 
“accredited, non-accredited or conditioned”, and from the result obtained, some financial, 
prestige or other consequence can be glimpsed. 


Therefore, it will be necessary to know if from the perspective of the public universities 
of Mexico (UPM) the evaluation processes are viewed as an opportunity for improvement or 
as a threat to academic diversity. And it is opportune to inquire, consequently, about the real 
incentives that Mexico has for academic evaluation: quality assurance, access to additional 
resources or prestige in the academic market? 


Through two thorough studies, Unique exam and culture of evaluation in Mexico 
and Critical issues of University in Latin America in the nineties. Comparative studies, 
Aboites (2002) and Kent (2020), formulated an analytical scheme around the features of 
the evaluation culture in Mexico. In considering the UPM, we agree with these authors 
in identifying, at least, the following weaknesses in the current evaluation system: 1) 
An evaluation process without comprehensive participation; 2) evaluation as a political 
discourse of opposition; 3) the absence of a legal framework for the current evaluation 
system; 4) the lack of transparency in the evaluation; 5) the lack of a meta-evaluation; and, 
6) evaluation as poor social policy. 


Therefore, it is to be observed in Mexican University a gradual acceptance of the idea 
of evaluation as an important component of what has generally been called modernization 
of University. However, its consolidation in institutional and academic practices and norms 
is very uneven, and even incipient. In universities and public technological institutes, the 
development of evaluation is hesitant and generates negative effects (Kent, 2002: 285-290). 


Consequently, from the foregoing it follows that the particular culture of evaluation, 
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which in fact proposes the official project of educational modernization, is incipient to build 
a project for a democratic country, founded on a culture of knowledge and participation, and 
prosperous for all. 


In the following spaces, the critical points referred to in Mexico, in our opinion, the 
current PU evaluation system will be addressed. 


1.1 Evaluation without Comprehensive Participation 


In the discourse of evaluation in Mexico, the absence of a clear and structured 
development that specifies the importance and function of the participation of all the 
actors (executive, judicial, legislators, secretaries of state, unions, rectors, academics) is 
notorious., scientists, students, experts and the general public). In the evaluation proposals, 
in their general scrutiny and in the specific forms that must be adopted, the unilateral vision 
of the Executive predominates. 


This has to do with the fact that educational evaluation in this country is not a matter 
of society but an act of the federal executive, despite the different organizational formulas 
it adopts. The evaluation also collects the less participative tendencies of the exercise of 
governmental power in the country; education, seen this way, is not a compilation of different 
and advanced practices. 


lfin the first education systems the obligation of the State was clearly to impart, at 
present it only corresponds to propose and attend; that is to say, an obligation only of a 
moral nature. 


For Kent (2002), the interest of the different political actors in Mexico has been poor, 
with the exception of the Executive power, experts, scientists and public opinion. And the 
claim of society is clear; however, the Legislative power cannot even be said to exist or be 
participating in decisions concerning academic evaluation. 


Actor / Country Argentina Brazil Chili Colombia Mexico 
Ministries Medium Medium Tall Bass Tall 
Legislators Bass Bass Medium Bass It does not count 
Unions Bass Bass Does not It does not count Bass 

count 
Students Bass Bass Medium It does not count Bass 
Rectors Medium Tall Tall Bass Medium 
Experts Tall Tall Tall Medium Tall 
Scientists Medium Tall Tall It does not count Tall 
Public opinion Tall Tall Tall itis not known Tall 


Table IV.1 Levels of Interest in the Evaluation by the Actors. 


Source: Kent (2002: 312). 
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Finally, an interesting piece of information provided by the study by Kent (2002) lies 
in the indifference that exists in the student body in most of the Latin countries considered in 
the aforementioned author's analysis, and which is available through Table IV.1. Therefore, 
the fact that the evaluation process in Mexico lacks comprehensive participation is endorsed, 
which constitutes one of the weaknesses of the current evaluation system. 


1.2 Evaluation as Contrasting Political Discourse 


A second feature of the Mexican evaluation scheme is an evaluation as a political 
discourse of opposition: The axis that opposes the light and the dark, that is, the legitimate 
interest in the transformation and improvement of educational quality on the one hand, and 
the dark motivation or illegitimate on the other. Thus, criticism or opposition to evaluation 
initiatives may well come from those who are systematically enemies of everything that 
comes from the authorities or worse, from the fact that the debate around these issues 
(evaluation) sometimes it hides another type of situation, such as the impossibility of calling 
those who evaluate to account, the evasion of responsibilities and the fear of facing evidence 


of inefficiency, insignificance, irrelevance or inefficiency. 


A second axis of opposition in the discourse is that of chaos-order. In this plane, 
apparently an alternative vision, about whether it is linked to an important mobilization, “is 
exaggeratedly treated as true chaos and about to happen” (Aboites, 2002: 125). Thus, the 
emergence of scholarships and incentives were surrounded by a discourse that apparently 
served to justify their structure. “The stimuli are not for everyone because not everyone 
works” (Ídem).! 

And a third axis of opposition is that of knowing-not knowing: The evaluator knows, 
that is why the problems, if any, are the result of the lack of information of those evaluated or 
of those in charge of operating the program, or they are the result of ignorance or personal 
or institutional incapacity.? Finally, the tendency to move in the discourse of opposition 
means a display of defense and an attempt to defeat the adversary. Such a discourse, 
therefore, is a very poor instrument if what you want to analyze and diagnose in depth are 
the problems of evaluation.? 


1 Aboites points out that: “The idea of allocating funds only for 30% of academics arose because, as we all know, many 
people do not work. Those who work, those must be rewarded” (Aboites, 2002: 125). And adds Kent: “A part of the aca- 
demics was incorporated into the evaluation processes and the corresponding incentives, but the salaries of the majority 
continued to be depressed. Thus, the selective increase in academic emolumenits (...) ran the risk of becoming primarily 
a new mechanism for segmenting the academic market and only secondarily an instrument for raising productivity (...) it 
is worth asking if this mechanism induces an increase in performance individual or if it is merely an instrument of diffe- 
rentiated salary increases” (Kent, 2002: 286). 

2 Kent points out that: “The accreditation offered (...) is not equivalent to a legal action but in view of the proliferation of 
private institutions of very different levels of quality under the traditional regime of lax government regulation” it exerts 
“an influence on the prestige and the public image of private institutions that had an impact on the consumer market” 
(Kent, 2002: 289). 

3 According to Kent: “lt is evident that there was a first politically flexible movement towards the installation of the me- 
chanisms and structures of evaluation in a system historically alien to it. However, their legitimation and full assimilation 
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Finally, the fact that the evaluation process in Mexico is linked to oppositional political 
discourse is endorsed, which constitutes one of the weaknesses of the current evaluation 
system. 


1.3 Absence of a Legal Framework for the Current Evaluation System 


The third feature of the current evaluation culture in Mexico is the lack of a legal 
framework. In other words, the current accrediting system, in addition to being a protected 
monopoly (Aboites, 2002), It does not have a legal framework that supports it.” 


In the Mexican context, the term accreditation, for example, does not appear in 
the law, although it is contemplated in the essence of the law, since it is an evaluation of 
educational programs. The current law does not contemplate subjecting University schools 
to accreditation. 


In the case of the Autonomous Public Universities (UPA), the status granted to them 
by Section VII of Article 3 of the Constitution is violated.? The same ANUIES promoter of 
such a system, asks the Congress of the Union to give legal powers to COPAES.” 


Finally, the accreditors as civil associations under the Federal Civil Code, they open 


into institutional and academic practices were uneven. There were mixed messages, a great diversity of government 
intentions and an uneven assimilation of the culture of evaluation” (Kent, 2002: 289-290). 

4 The civil associations in the formality are private organizations; however, civil associations for the external evaluation 
of the IESPs have among their members the SEP, the ANUIES, public and private universities, civil associations of 
professionals and academics, and companies. This is the main justification that they are not private organizations in 
practice; however, it is precisely the main weakness to assert that they are autonomous organizations, a necessary 
attribute for external evaluators. The SEP allocates additional financing for the IESPs that are accredited through these 
associations (in which the SEP itself participates) making it difficult for the government to audit the donations it makes for 
this purpose. On the other hand, associations such as CENEVAL, sometimes charge a fee to students for applying an 
entrance or exit exam, in order to comply with part of the procedures in the IPES, which contrasts 

with Article 3 of the Constitution: “All education provided by the State shall be free.” In addition, these civil associations 
exert a great influence on the course of academic evaluation, a monopoly of a power that must only be exercised by the 
Executive and Legislative powers. 

5 The General Law of Education clearly states, in its Article 29, that: “It corresponds to the Secretariat (of Education) the 
evaluation of the national educational system, without prejudice to the one that the local educational authorities carry 
out in their respective competences. Said evaluation, and that of the local educational authorities, will be systematic and 
permanent. Their results will be taken as a basis for the educational authorities, within the scope of their competence, 
to adopt the appropriate measures”. And it continues: “Educational authorities will make known to teachers, students, 
parents, and society in general the results of the evaluations they carry out, as well as other global information that allows 
measuring the development and progress of education in each federal entity” (Articles 30 and 31 of the same Law). (H. 
Congress of the Union, 2001). 

6 The aforementioned article establishes that: “Universities and other higher education institutions to which the law 
grants autonomy, will have the power and responsibility to govern themselves; they will carry out their purposes of edu- 
cating, researching and disseminating culture in accordance with the principles of this article, respecting the freedom of 
academics and research and the free examination and discussion of ideas; they will determine their plans and programs; 
they will set the terms of entry, promotion and permanence of their academic staff and will manage their assets”. (H. 
Congress of the Union, 2001). 

7 Part of the ANUIES agenda has to do with the idea that: “For the future development of higher education it is imperative 
to have an adequate legal framework (...) and the establishment of an accreditation scheme for academic organizations 
and programs with Official validity”. (ANUIES, 2000: 250). 

8 In its Article 2670, the Federal Civil Code establishes: “When several individuals agree to meet, in a way that is not 
entirely transitory, to carry out a common purpose that is not prohibited by law and that is not predominantly economic in 
nature, they constitute an association.”. (H. Congress of the Union, Mexico, 2001). 
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the door to suspicion and the possibility of acts of corruption, given their discretion to operate 
and the discreet and indirect support of the Executive branch.” Additionally, academics, 
trade unionists, students, and parents expressed their concern in a letter addressed to the 
Secretary of Education, due to the operation of a civil association (CENEVAL, A.C.) with 
functions that are exclusive to institutions.'º The issue of legality is important in terms of the 
construction of a different political culture framed not only by government purposes, but also 
by the rights of individuals -student academic workers, applicants for education, parents, 
communities and organizations- to have a guarantee that the actions of the Executive 
Power are not surprising, arbitrary, harmful and unappealable for preponderant sectors of 
the population. And it is precisely their private nature that allows civil associations to remain 
outside any public scrutiny and regulation that seeks to regulate their functions. 


Therefore, in Mexico, another feature of the evaluation culture has to do with the 
absence of a legal framework for the current evaluation system, precisely, which contributes 
to the weaknesses of said system. 


1.4 Lack of Transparency in Evaluation 


A fourth feature establishes an evaluation with a lack of transparency: The culture 
of evaluation in Mexico shows little concern for informing the public of the arguments that 
justify the design of mechanisms, the transparency of results, and the decisions that will be 
made before the inefficiencies. 


For example, in the case of CENEVAL, the SEP does not give explanations because 
the single exam as an evaluation mechanism excludes the certificate of studies, vocational 
orientation interviews, preparatory courses, among others. In addition, the results of said 
evaluation are not reported, as well as the solutions proposed to the inefficiencies exposed. 
The final assignment of students to the IESPs depends on the combination of correct 
answers and preference order. Attempts to make “corrected” tests that resolve these biases 
have apparently failed to get rid of important features (Aboites, 2002). 


Due to arguments such as the previous ones, it is seriously questioned whether the 
option test multiple is an instrument capable of predicting the future performance of the 
student and if it is not an abuse to use it as the main or as the only indicator to determine 
admission to an institution, the granting of a scholarship, among others. Issues capable of 
seriously and permanently affecting the lives of young people. 


9 Such as onerous salaries, negligent administration in financial control, purchase of unnecessary material resources in 
exchange for “bribe”, nepotism, personal favors such as the issuance of professional titles, among others (López, 1998: 
20, 24, 29 and 30). 

10 In the aforementioned letter delivered to the SEP on July 24, 1996, addressed to its owner, it is established that: 
“The decision to put in the hands of CENEVAL, the core part of the conduct of the process has represented that in fact a 
private entity performs functions that are the responsibility of the institutions and the secretary itself (examination design, 
validation, registration organization, payment, exam qualification, school assignment, resolution of disputes about the 
process, such as certificates, among others). With this, applicants are disadvantageously subject to acts whose authority 
is not clearly established or strictly linked to the proposed purposes. (Aboites, 2002: 130). 
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Hugo Aboites mentions that the use of the school average as part of a selection 
mechanism coincides with a trend already present in other countries that seeks to leave 
standardized exams behind, replacing them with other forms of evaluation. At the same 
time, the idea that multiple choice tests are not reliable is gaining social consensus.'! This 
same author considers that the culture of evaluation is one of the most powerful factors - as 
much or more than the administration of financing - capable of guiding education in one 
direction or another.!? 


In the case of the evaluation of institutions, the CIEES do not make public the results 
of the evaluations, it is exclusive information of the rectors. In the case of COPAES, it carries 
out evaluations based on a manual similar to a check list, but does not argue its design that 
justifies classifying study programs or institutions as quality. 


Thus, in Mexico the culture of evaluation is presented as subtracted from a proposal 
of clarity, transparency and accountability. And this, finally, is one more of the weaknesses 
of the current evaluation system. 


1.5 Lack of a target-evaluation 


Finally, there is no meta-evaluation, that is, the evaluation of the external evaluators. 
Evaluators have a professional obligation that proposed or completed evaluations be subject 


to competent evaluation, the meta-evaluation. 


The rationale for meta-evaluation is that evaluation is a particularly self-referential 
subject, since it applies to all serious human endeavors and, consequently, to evaluation 
itself. In other words, that evaluation “begins at home. The objective of meta-evaluation is, 
likewise, to ensure the quality of evaluation services, to avoid or confront illegal practices 
or services that are not in the public interest, to point the way for the refinement of the 
profession and promote a greater understanding of the evaluative enterprise (Stufflebeam, 
1974). 


The meta-evaluation allows assessing the quality with which the evaluation process 
was developed and the sufficiency of the evaluation system; ratifies the qualities or defects 
of the programs that have been evaluated; and determines the strengths and weaknesses 
that were not detected by the primary evaluation. In addition, it ratifies or not the principles 
on which the widespread evaluation policy is inspired, that is, the relevance, fairness and 
participatory nature of the evaluation. 


Finally, in Mexico the evaluation scheme is presented without meta-evaluation, that 
is, there is a lack of a process that investigates the weaknesses and strengths that mark the 


11 One example is a judge's ruling in favor of a young woman who sued the State of New York for denying her a scholar- 
ship based on the results of one of these tests. The judge considered the exclusive use of a single test for this purpose 
to be unfair (Aboites, 2002: 136). 

12 Thus, “more than the philosopher, the teacher or the organizer of study plans and curricula, the evaluator is capable 
of sending a very clear message about what is important in the educational process. In fact, he is also capable of saying 
what It is important to build the future of the country”. (Ídem). 
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evaluation processes, ensuring compliance with the ethical, methodological, regulatory and 
technical aspects that govern each process and whose results are likely to be considered 
for the projection of the subsequent improvement of the evaluation practice, in general, and 
of the evaluated programs, in particular, with the aim of raising their quality. 


1.6 Evaluation as Poor Social Policy 


In general, in the Latin American educational system there are very few experiences 
of academic evaluation that start ffom a broad conception of the political, academic and 
theoretical dimensions that underlie the practice of evaluation. The current evaluation 
programs encourage competition in the region, but do not promote collaborative actions 
between students, teachers and educational institutions. 


The formative evaluation starts from another conception, on the pedagogical 
processes and seeks to detect the difficulties and deficiencies of educational work in the 
process itself, so that the improvement and modification occur before the educational task 
is finished. For this reason, formative evaluation seeks to generate certain participation 
processes, or at least an evaluation practice is sought in which users are the main recipients 
of the results; promotes elements that enable a better understanding of its operation; 
estimates that, in life, subjects need to develop analysis models in the face of diverse 
information; and at no time are the teachers classified (as good or bad), nor the students, 
on the contrary, the shortcomings in various school processes are addressed. 


The idea of this perspective has been to contribute to the improvement of the 
educational process, showing how it works and allowing the members of an educational 
institution to have access to that information (Díaz, 2002: 28). 


International organizations and educational policy designers mistrust formative 
assessment. Thus, they avoid promoting it because their moral, political and scientific 
commitment links them much more to evaluation processes as judgments (of quality and not 
of quality), which shows to a certain degree their disconnection with educational planning. 
The existence of suspicions and mistrust in the judgments due to the lack of participation 
and information lead to academic statements about adverse behavior in the functioning of 
the educational system. Not all programs that pass an academic certification deserve their 
accreditation, nor do the excluded ones deserve their exclusion. The same happens with the 
grade given to researchers or professors.!º 


The evaluation schemes adopted by the country, far ftom promoting experimentation 
to generate their own proposals, have accepted only formal strategies that allow quantifying 
educational results, adequate for accountability, but that prevent knowing the process and 
the academic quality of the process. Thus, the Mexican evaluation rates people (students- 


13 In the case of students, the qualification -as a result of an admission or placement exam- acquires dramatic overtones: 
The subject's life depends on them. The improvement of the educational system demands the adoption and development 
of an evaluation paradigm linked to formative evaluation, without giving up accountability. 
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teachers) and institutions, but does not rate the academic process that underlies educational 
processes. 


Therefore, an error of the Mexican educational policies has been to inefficiently 
link, between mixing objectives, the results of the evaluation to a budget allocation.!* 
The programs for teachers (teaching career or incentives for academic performance), for 
researchers (SNI) and for institutional development (Fund for the Modernization of University, 
Postgraduate Register of Excellence, among others), sooner or later culminate in a budget 
bag. Although people and institutions strive to meet the requirements that allow them to 
access such economic resources, their effort does not mean that they take consistent steps 
to improve academic work processes.!º The design of these incentives has a purpose that 
is not easy to fulfill: Align the interests of the professors, researchers and administrators of 
the IESP with the public or State interest (Hamilton, 1998: 220 and 308). 


Therefore, it is emphasized that education is a complex process; a student not only 
learns knowledge in the classroom, he also develops ways of thinking, of valuing, he also 
integrates as a person and as a citizen.'* Thus, there is an absence of evaluation conceived 
as an efficient social policy. A culture of evaluation and accountability in education would 
mean that all members of the educational community; students, teachers, administrators, 
representatives, parents, or other sectors of society find valuable access to information on 


educational performance as well as naturally seeing evaluative actions. 


Acquiring a culture of evaluation means seeing, for example, with pleasure the visit 
of supervisors to schools, not fearing the application of exams at any time and carrying out 
self-evaluations with pleasure. This type of description would only be achieved if instead 
of justifying and finding culprits, the analysis is focused on resolving the deficiencies 
found. It must be clarified that accountability structures are not incompatible with formative 
evaluation, but they do express the need to make a link with educational planning, in order 
not to fall into contradictions.!” 


14 And itis that, according to Kent, on the one hand “they sought to put an end to anarchic bureaucratization: streamline 
school administration, improve financial and human resource management, make planning effective and adapt the legal 
system of institutions. On the other hand, the evaluation programs intended, at least on paper, to introduce the norm of 
external guarantee (accountability) and to ingrain the habit of continuous improvement. However (...) the two purposes 
seemed to intermingle, with which the principle of evaluation ended up being diluted” (Kent, 2002: 289). 

15 In this regard, Kent observes that: “The creation of an evaluation system always faces an underlying dilemma. It can 
be contributed by tying funding to evaluation results in order to ensure that evaluation is effectively introduced. The risk is 
that evaluation procedures become a game for power (stimuli for academic performance). If, on the contrary, it is decided 
that the fundamental objective is to promote interest in the evaluation freely assumed by the actors, it will be thought 
that the evaluations must not have financial consequences. The likely cost of these strategies is that quality assessment 
not be effectively incorporated into institutional and individual practices (failure of CONAEVA)”. (Kent, 2002: 287). 

16 In this regard, Díaz affirms that: “The school must open itself to the world of work, but it must also attend to what has 
been its historical function: to bring culture closer, to bring national knowledge closer, to enable human encounter and 
development.” (Diaz, 2002: 30). 

17 According to the SEP: “Good quality implies evaluation. Evaluation is conceived as an indispensable means for 
continuous improvement and quality assurance, as well as for accountability. In addition to evaluating, it is essential to 
make the results known and use them for decision-making.” And it continues with the vision to 2025: “Equity with quality, 
quality with evaluation, evaluation with accountability, and accountability with social participation” (Ministry of Public 
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The need for accountability expresses, ultimately, the distrust of citizens towards 
public officials. Thus, representatives and other public officials are mistrusted and rules 
(institutions) are established with the purpose of establishing trust. Without this basic 
structure, any evaluation will be linked to suspicion and discretion in handling money of 
public origin, which would mean polarization in society, expressions of grievance, and in the 


worst scenario, radicalization. 


Finally, the description and analysis of the main limitations, formal and informal, of the 
evaluation and accreditation system of the IESP, as well as the definition of accountability in 
the institutions, have provided important elements for the design of an outline of academic 
evaluation proposal as a public policy instrument in favor of educational quality and 
accountability. 


21 SCHEME OF ACADEMIC EVALUATION OF THE PUBLIC UNIVERSITIES 
OFMEXICO WITH A FOCUS ON QUALITY ASSURANCE AND PERFORMANCE 
OF ACCOUNTS: A PROPOSAL 


2.1 Academic Evaluation, Quality and Accountability 


As it has been reiterated here, the quality of teaching and research has become 
one of the most important issues for governments. The world is changing rapidly, as it is 
increasingly competitive and uncertain. For this reason, all institutions, including universities, 
have increased their efforts to respond to society's needs in an efficient and timely manner. 


One of the main reasons for this trend is the value of money, a perspective of both 
representatives (rulers) and taxpayers (citizen-consumer), a reason that motivates citizen 


demand to be accountable in terms of quality and public sector efficiency. 


For the above reasons, it is legitimate to require those responsible for universities 
and research and development centers to render accounts and make their management 
transparent. They are required to meet a minimum standard of quality, in addition to making 
a constant effort to improve education. It is clear that the second objective must be a 
permanent concern of the universities, if they want to be in a good position to compete. 


Basically, any evaluation effort can be implemented from two points of view. The first, 
which is external, is normally carried out by governments and by national and autonomous 
agencies, where the purpose is to control and/or measure. It can be generated in different 
ways: Accreditation, institutional evaluation and benchmarking.18 

The second point of view is internal, and is carried out by the organization of the 
university. Internal evaluation seeks that the governance and quality improvement systems 
Education, 2001: 236). 


18 Anglo-Saxon word meaning point of reference (Goldsmith and Pérez, 1997: 342). Evaluation institutions do not carry 
out Benchmarking, it is usually developed by specialized magazines (Weber, 2003: 2). 
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of an institution concentrate their efforts to improve the quality of teaching, research and 
specific policies (internationalization, resource management and student support). Such a 
process is normally articulated by a self-evaluation in collaboration with external visitors 


(external evaluation) and the evaluation of courses or programs. 


In the following spaces, a proposal will be presented that responds to the quality 
assurance and institutional accountability approach that has been proposed here as an 
imperative. 


2.2 The Institutionalization of Evaluation: Outline of a Proposal 


Below are some suggestions for the academic evaluation of IESPs in Mexico, based 
on efficiency (quantitative methods) 19 and formative or pedagogical evaluation (quantitative 
and qualitative methods),20 linked to accountability structures.21 


The proposal focuses on a national evaluation system based on the powers that 
the H. Congress of the Union has to supervise the work of the Executive. The Evaluating 
Institution (El) that is proposed here, should first have specialized commissions in the 
different areas of knowledge, in order to maintain scientific parity, the commissions will be 
formed by specialists in the area. 


In addition, it will be necessary to design an Organic Law 22 to govern El, and to 
voluntarily involve IESPs in internal and external evaluations. The evaluations must be 
published so that citizens know the results of the IESPs, which would help new students to 
choose the career and school that meets their expectations. 


The university must publish the goals and objectives of the institution, the selection 
procedures for directors and commissioners, methodologies used in the evaluation, 
scientific and pedagogical foundations for the design of indexes, results, recommendations 
issued, the financial statements of the Institute, as well as the response of the IESPs to their 
recommendations. 


Transparency must be characteristic of IE in order to generate a reputation that 


19 According to Puente, “the efficiency approach (quantitative method), understood as the efficiency of the system in 
terms of spending and performance... society does not dispute spending on education, it does question how. For this 
reason, direct or indirect factors of the efficiency of the educational system are evaluated, such as the management of 
the centers, teacher training or the impact of the media. There is consensus that efficacy does not guarantee quality: It 
is a necessary condition, but not sufficient” (Puente, 2004: 3). 

20 And, the formative or pedagogical evaluation (quantitative and qualitative method), for its part, “focuses directly on 
the improvement of the quality of the evaluation. Under this approach, it is sought to know the quality of the educational 
service, monitoring for this the role played by the different components of the system: The educational administration, 
the management and organization of the centers, the teachers, the students, the curriculum, the didactic methodologies, 
among others. This approach is always present an issue for controversy: the different opinion about the concept of qua- 
lity have the administration, teachers, family, among others.” (dem). 

21 The agent must not only report the results of his management, he must also report on the decisions made in this 
regard (Ugalde, 2002: 14). 

22 The law will describe, among other aspects: Objectives, ends and goals; organizational structure (Council, specia- 
lized Commissions, Appeals Commission, among others); own resources; and information and transparency (Detailed 
Reports and General Reports). 
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builds authority in the academic environment. Being an institution to which responsibilities of 
the H. Congress of the Union are delegated, the counselors of this institution (coordinators 
of this) must be accountable to it. The council of the IE will be chosen by the Legislative 
power, although the shortlist for councilors could be proposed by the SEP and the ANUIES, 
as long as they are citizens without party affiliation or minister of any religious cult. And it is 
of vital importance that the directors remain in their positions continuously, unless Congress 
considers their removal, because this way the planning of the institution around the task of 
evaluation would be given continuity. 


On the other hand, economic incentives from the government must continue. Howe- 
ver, the evaluation must be aimed at defining training programs, as well as deficiencies in 
the school infrastructure. The evaluations must not be used to violate the right to education, 
discredit the work of the IESPs, or to generate monopolies, as in the case of COPAES and 
CENEVAL. The evaluation with a diagnostic approach is appropriate, since an “accredited 
(quality) and non-accredited (non-quality)” solution is not sought, which could well harm the 
IESPs, before benefiting them. 


Also, in order to reduce evaluation costs, it is suggested that each of the IESPs 
have a Social Council (SC), which would be in charge of carrying out the internal evaluation 
under the supervision of the IE, this way institutionalizes the Internal Evaluation / External 
Evaluation relationship. Another of the functions of these councils will be to express 
disagreements of their universities before the possible recommendations. Appeals are 
valuable instruments, as long as they have scientific and pedagogical bases, to improve the 
evaluation methodology. And, must the appeals proceed, El would be obliged to rectify its 
recommendation. 


The answer to the question, who watches the security guard?, is currently in the 
Superior Audit Office of the Federation (ASF), which in turn is supervised by the Surveillance 
Commission of Congress. And, additionally, the supervision or monitoring will be carried out 
by the SEP? Therefore, both the honesty and the performance of the evaluation institution 
for the IEPS will be monitored. 


The proposed mechanism of accountability through academic evaluation coincides 


23 The education commissions of the Chambers of Deputies and Senators have the power to call to account the Pre- 
sident, the Secretary of Public Education, and for the proposal to the Council of the Evaluating Institution, according to 
the Organic Law of the General Congress of the United States Mexicans (H. Congress of the Union, 2003). According 
to Montesquieu, Congress must not have powers of execution, because “when the legislative power and the executive 
power meet in the same person or the same body, there is no freedom; confidence is lacking, because it can be feared 
that the monarch or the senate will make tyrannical laws and execute them tyrannically themselves (...) The representa- 
tive body is not elected (...) to make any active resolution, which it would not do well, but to make laws and to supervise 
the faithful execution of those that exist”. (Montesquieu, 1998: 104 and 106). 
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with the definition described by Crespo (2000) and Ugalde (2002).2º Thus, accountability 
is carried out vertically, that is, citizens are informed, so that their decision-making at the 
time of exercising their vote for their representatives, is carried out with greater certainty. 
In addition, its complement, horizontal accountability, is fulfilled, that is, under a system of 
mutual surveillance based on the division of powers. 


2.2.1 The Institutionalization of Evaluation: Elements and Objectives 


In general, among the elements of accountability are: The monitoring of honesty 
(legal responsibility) and effectiveness (political responsibility) of public servants; the 
publication of the results derived from the presentation of information by the public servant 
or by monitoring (audits and evaluation) and the assignment of a punishment (criminal, 
administrative sanctions, electoral punishment, among others) or an award (recognition, 


incentives financial, electoral prize, among others), depending on the results. 


Thus, the elements of accountability are built vertically and horizontally, that is, in the 


division of powers and in a system of democratic representation. 


In the case of public University in Mexico, accountability is not exempt ffom meeting 
its two objectives: honesty and efficiency. As far as honesty or legal responsibility is 
concerned, in Mexico there is the ASF, in addition to the institutions of budgetary control by 
the Executive (Secretariat of Public Administration, Ministry of Finance and Public Credit, 
among others). And, in the case of performance, there is only the evaluation of the SEP as 
an institutional figure. The SEP (Executive power) and the ANUIES, as has been pointed 
out in advance in these spaces, have created a series of private organizations that have 
been called the evaluation and accreditation system, although none of these organizations 


24 According to Crespo, accountability must meet certain assumptions, so that one can speak of the existence of an 
effective political democracy: A) That all rulers, at all levels, are subject to accountability. In a democracy, whoever has 
more power has greater political responsibility. That is, the most powerful with the greatest reason must be able to be cal- 
led to account for their decisions. B) Accountability must proceed from the bottom up, that is, it is the representative insti- 
tutions themselves, the citizens and the courts, who must be able to call the rulers to account at different levels, including 
the head of government. If accountability does not flow from the bottom up, but vice versa, then the head of government 
would go unpunished. C) In a democracy, the political institutions will have the ability to call the rulers to account, but in 
a peaceful manner. Accountability provides information to interested citizens about the efficiency of public organizations 
that must meet the explicit objectives of the institutions (Laws). Therefore, accountability is made up of three elementary 
institutional figures: 1) Evaluation (internal and external) of public organizations; 2) Publication of the evaluation results; 
and 3) Mechanisms of reward and punishment of citizens towards the public administrator. The three institutional figures 
for accountability are formed in a system of circular powers, a system that has been called counterweights. In the case of 
Mexico, there are three important powers: the executive, the legislative and the judicial, each of which has surveillance 
mechanisms for the other two powers. Accountability is aimed at evaluating the honesty of public administrators, as well 
as the effectiveness of their work. The types of governmental responsibility that have been described are synthesized 
there: Legal, related to honesty, and political, associated with efficiency (Vid. Crespo, 2000: 6-55). 

25 Under the institutional approach, Ugalde provides, in his definition of accountability, a synthesis of work carried out by 
various researchers: “Accountability is defined as the permanent obligation of the leaders or agents to inform their princi- 
pals or principals of the acts that they carry out as a result of a delegation of authority that is carried out through a formal 
or informal contract and that implies sanctions in case of non-compliance. The principals or principals also supervise the 
principals or agents to guarantee that the information provided is reliable” (Ugalde, 2002: 14). 


Synthesis around the Evaluation of the Quality of Education in the Public Universities of Mexico: Criticism and E 
Proposal 


can be considered within the rendering of accounts, because being private, the State cannot 
impose the interests of the nation and, therefore, cannot demand that they render accounts 
to the citizenry. 


In summary, Mexico does not have horizontal accountability for the effectiveness of 
University, it must even be clarified that the other levels of education do not have horizontal 
accountability either. 


Inthe context of the scheme proposed here, when making an approach to the possible 
objectives that academic evaluation would have as an instrument of accountability in public 
University, three sources are identified for the description of the objectives: 1) Those of 
legal and constitutional origin; 2) those caused by scientific, technological and pedagogical 
arguments;? and 3) those that are originated by the principal-agent relationship.” 


In the first place, through the objectives of legal and constitutional origin, it is 
necessary to seek to evaluate or monitor the performance of the IESP before the objectives 
of education that are mainly described in Article 3 of the Constitution and in the General Law 
of Education.” Second, the objectives that are originated by scientific, technological and 
pedagogical arguments identify that the evaluation of the training of university professionals 
originates in light of the set of modifications in the institutional, cultural, scientific processes 
involved and in the labor market. And, in third position, the objectives originated by the 


26 Vid. Pacheco, 2002: 90-109. 

27 Vid. Ortmann and Squire, 1996: 9-25. 

28 For the Legislative power, the Constitution dictates the responsibility of issuing the necessary laws, aimed at distri- 
buting the educational social function among the Federation, the States and the Municipalities; set the economic contri- 
butions; and point out the sanctions applicable to officials who do not comply or do not enforce the relative provisions, 
as well as to all those who violate them. In addition, the Constitution dictates to the Executive an education based on 
human rights, secular, scientific and cultural with a total subsidy in the education provided by the State and specific attri- 
butions for the autonomous universities and higher education institutions. On the other hand, it dictates to Congress the 
allocation of budgets and the monitoring of both the honesty and the effectiveness of the public servants who administer 
the educational system. 

29 In neo-institutional political science, an analytical ffamework has been developed called the agent-principal model that 
implies the process of delegation and the accountability that accompanies it (Ugalde, 2002: 19). This model has been 
studied by various academics specializing in game theory (Cf. Rasmusen, 1996: 195; and Shubik, 1992: 613 and 1996: 
367). This game is about a relationship between a subject (called principal) who delegates to another subject (called 
agent) authority to execute acts on his behalf. In exchange for this delegation, the agent agrees to render accounts 
(signs) to the principal, who has the power to sanction him in case of non-compliance or remove him from the position in 
case of poor performance. One of the most important aspects in this game is the asymmetric information that generally 
exists between the principal and the agent (Rasmusen, 1996: 195-199). Asymmetric information can lead to problems in 
the principal-agent relationship, as the agent has incentives to deviate from the principal's mandate and act for his or her 
own benefit. To solve the problems of opportunistic behavior, some responses have been developed that seek to mitigate 
the opportunism of agents against their principals, of which the most important due to its effectiveness is accountability 
(Rasmusen, 1996: 171-185; and Ayala, 2003: 167-172). In order for the principal to be able to offer incentives to its agen- 
ts when they fulfill their mandate and sanction them when they do not, itis necessary to have accountability systems that, 
on the one hand, oblige the agent to report his acts to the principal in detail and results and, on the other hand, provide 
the principal with mechanisms to monitor the performance of the agent. Finally, the principal must have punishments to 
sanction the agents who did not adequately fulfill their mandate. Without sanctions, the principals are powerless against 
their agents. In political matters, the punishments (formal and informal) include criminal and administrative sanctions, 
violations and electoral punishment at the polls during the next election. 
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agent-principal link,*º must be aimed at evaluating the reputation of the IESPs related to a 
culture in science and technology, in order to increase the economic value of the certificate 
or title of the graduates and to improve the dissemination of the university culture (science, 
technology, fine arts, sport, among others).º' 


2.2.2 The Institutionalization of Evaluation: Accountability 


Accountability is a global concept that accepts classifications according to various 
criteria. The most suggestive classification of accountability was developed by O'Donnell 
(2000: 7), who divides this concept between horizontal accountability and vertical 
accountability. 


Horizontal accountability refers to the existence of state agencies with legal authority 
to take punitive actions, in relation to illegal acts or omissions of other state agents or 
agencies (Ugalde, 2002: 27). Surrender is horizontal insofar as it develops between 
institutions at the same hierarchical level, but independent of each other. 


30 Ortmann and Squire (1996) present a game-theoretic approach to the internal organization of IESPs, suggesting that 
higher-level learning institutions can be conceptualized as a principal-agent cascade, which describes a set of games 
in a vertical structure of agents (subordinates) who are hired by principals. Accountability allows the last principal (pupil/ 
student) to reward or punish the first agent (supervisor/representative) of the cascade (Ortmann and Squire, 1996: 9). 
And, for his part, Shubik (1992) has described public administration as a game of games (Shubik, 1992: 614) 

31 The Ortmann and Squire cascade runs through four levels, with each level taking the form of a player who represents 
one of the four constituents of the IESP. Students/Students play the role of the ultimate principal, Teachers take the role 
of the ultimate agent, and Supervisors/Representatives and Administrators take both agent and principal roles, depen- 
ding on their position in the cascade. In order to detect the goals and interests of each player, Ortmann and Squire carry 
out a brief analysis of each player: 1) Student: The player is interested in improving the real and perceived value of the 
IESP diploma, for which he demands your agent (supervisor/representative) increase the reputation of the SEE and hire 
efficient administrators and teachers. 2) Supervisor/representative: The player's goals are to maintain and improve the 
reputation of the school, maintain the trust of the Student through the hiring of efficient administrators and teachers. De- 
mand of his agent (Administrator) to maintain the confidence of the Student, primarily by a responsible administration of 
the finances of the IESP. 3) Administrator: The Administrator has as goals: to increase the reputation of the IESP, increa- 
se income and delegate administrative duties to professors or support people or staff. Demand your agent (Professor) 
to increase the reputation of the school through publication and teaching; as well as sharing administrative duties with 
him. 4) Professor: Its goals are to increase his professional reputation; their permanence in the position; use his time 
in IESP activities, time that he considers as free time; increase income from abroad through marketing, consulting and 
conferences. Finally, from this cascade the following sets can be obtained: 1) Student/ Student-Supervisor/ Representa- 
tive; 2) Supervisor/ Representative-Administrator; and 3) Administrator-Professor. One of the solutions to the asymmetric 
and incomplete information that could exist in any of these games is accountability. Accountability, in addition to being 
a right, is a technological solution for the orientation of tensions, reducing conflicts, and at the same time improving the 
performance of government organizations. 

32 This way of rendering accounts presupposes the vigilance of the organs of the State by other institutions, also state 
ones, endowed with autonomy to exercise control functions. The Executive, Legislative and Judicial powers, in the dif- 
ferent spheres of government, must be accountable to each other and to other autonomous institutions whose task is to 
review their behavior. 

33 In horizontal accountability, the Executive, Legislative and Judicial powers, in addition to fulfilling their own functions 
(executing laws and carrying out a government program, creating and modifying laws, and ensuring the legality of acts 
and compliance with the laws, respectively), they also have the responsibility to supervise each other based on the 
principle of checks and balances. If any power, theoretically and constitutionally, had the power to regulate the exercise 
of other powers, the principle of checks and balances would be weakened and horizontal accountability would be uni- 
directional and would end up facilitating the arbitrariness and discretion of the strongest power, which the same in the 
case of the Executive than the Legislative. And, the mutual surveillance between the Executive and Legislative powers 
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The vote of the voters, as well as the work of citizen groups and the media are 
instruments that complement horizontal accountability for the purpose of holding the 
government accountable. This is a second aspect, vertical accountability. This describes a 
relationship between unequals: The bureaucratic surrender in which a hierarchical superior 
tries to control his subordinates, or the electoral surrender in which the voters judge and 


monitor the representatives. 


And, according to O'Donmnells classification (2000: 7), vertical accountability is 
divided into two, electoral and vertical social. The first refers to elections as a mechanism to 
stimulate the responsibility of governments.** 


The other aspect of vertical accountability, called vertical social, is made up of citizen 
groups and the media. Unlike electoral suffrage, this mechanism rests on measures based 
on moral and public criticism.*s 


This accountability scheme applied to the academic evaluation process of the IESPs 
in Mexico will try to be reproduced in the following spaces. 


2.2.2.1 Horizontal Accountability 


The proposal complies with horizontal accountability, since an institutional figure is 
formed based on the division of powers. The Legislative branch has the institutional powers 


requires, on occasions, the intervention of the Judicial Power, which is constitutionally empowered to intercede when 
conflicts arise between powers. 

34 Although the citizen vote lacks binding effects and sanctions in the event that a ruler does not fulfill his campaign 
promises, it serves to warn the rulers and legislators that an incorrect exercise of the public function has consequences 
and this tends to stimulate the responsibility of the government. Voters (voters-consumers) can exercise sanctions by 
not re-electing the ruling party or punishing a representative by denying him a second term (Ugalde, 2002: 31). That 
well-established mechanism is constituted by regular, universal, impartial, credible and competitive electoral elections. 
Who watches the watchman?, an expression that synthesizes the theoretical problem of accountability, finds part of its 
answer in electoral suffrage. The vertical and rigid structure of the bureaucracy explains how obligations are delegated 
and accountability for actions related to job performance is demanded. But the pyramidal structure of the government 
ends up depositing in a single person the ultimate command, in which there is no subsequent chief. That position is the 
head of government, entrusted to a president or prime minister. The question then arises about who watches that boss. 
The answer, in addition to the checks and balances of the separation of powers, rests on the citizen vote. If the president 
or prime minister acts irresponsibly, the voter has the power to place another party in government at the next election. 
And if re-election to the head of government is a constitutional possibility, the vote is strengthened as a negative incentive 
to force the government to abide by the legal framework. The voter is the ultimate watchman. 

35 Exhibit and disqualify the government for certain actions constitute the body of the sanction. Those sanctions can be 
transformed into a punishment at the polls. Or, they can trigger oversight processes by Congress or the Judiciary to even 
conclude in a criminal or administrative sanction, typical of horizontal accountability (Ugalde, 2002: 32). Vertical social 
accountability includes various actions, exposes the government's mismanagement, introduces law reforms through its 
representatives, or influences certain public policy decisions implemented by the government. To promote these actions, 
the media, organizations and social movements promote legal mechanisms for monitoring public policy. In this way, ver- 
tical accountability mechanisms can stimulate the subsequent activation of horizontal accountability mechanisms. Unlike 
horizontal accountability, in which oversight has criminal or administratively binding sanctions, vertical accountability only 
has, in the first instance, moral or symbolic sanctions. There are no financial fines or imprisonment. Social organizations 
and the media carry out investigations, pronouncements and trials aimed at exposing the public servant who has lacked 
honesty, transparency, or has attacked freedom. However, both accountability systems, horizontal and vertical, are 
complementary. Legal and administrative actions are essential to prevent, sanction and, eventually, repair the damage 
caused by an illegal or corrupt act. And vertical accountability actions can trigger horizontal surveillance in a later period. 
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to establish the course of University, as well as to monitor the executive branch's compliance 
with the law issued in this regard; These powers are not held by the SEP, the ANUIES, the 
COPAES or the CENEVAL. Congress is the power of representation par excellence, it is 
through this power that society must send its institutional proposals. 


In Mexico, there is a mistrust of the educational system by officials of the SEP.ºº It is 
the SEP itself that rules out the possibility of using the average of the certificate of studies, 
which it itself issues for admission to the upper secondary and higher levels. Additionally, it 
creates with the ANUIES an evaluation and accreditation system with civil associations, which 
obtain income directly, since additional financing from the public treasury is instrumented to 
encourage evaluation and accreditation through these organizations. 


It has been pointed out that private organizations cannot be subjected to public 
scrutiny, much less if they are civil associations, for the simple fact that they are private, 
these signs are adequate (at least while the current federal civil code remains). It is the 
SEP that has exceeded its powers; however, the responsibility is not only of the Executive 
power; The Legislative Power and the Judicial Power are also responsible for this diversion 
of powers, since their indifference or inefficiency is what has allowed the Executive Power 
to be the only one that dictates the scientific-cultural direction of University. 


2.2.2.2 Vertical Accountability 


The proposal that is poured here, complies with vertical accountability by 
providing for the publication of the results of the evaluation (diagnostic type); as well as 
the recommendations issued by the IE. In such a way that the student may have more 
elements to make their decisions with greater certainty, in the choice of the study program 
and educational institution. 


The citizen-voter-consumer *” You will have information on the performance of public 
servants in University, therefore, you will be able to exercise your right, with greater certainty, 
to electorally punish or reward political parties and representatives. It is clear that this type 
of accountability is only in the form of pulses, that is, it does not originate until after a long 
period of administration of power, however the influence it can generate is as high or greater 
than accountability. horizontal accounts. 


Itis essential that Mexican society once again trust in institutions. In this sense, North 
(2001: 139-152), Nobel laureate in economics in 1993, has warned of the consequences of 
not paying attention to the design of institutions that seek dynamism in stability. The difference 
between northern and southern countries, according to this author, is the responsible 


36 According to Noguez: “Secondary teachers complain about how unprepared primary school graduates are. At the high 
school and professional level, the deficiencies of first-year students are also criticized. It seems that the educators are to 
blame, right? Because no one can blame them anymore, except the family”. Noguez, 2004. 

37 Vid. Shubik, 1992: 607. 
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design of institutions. The indifference to the knowledge and design of formal and informal 
limitations produce social costs that cause uncertainty, distrust, low cooperation, acts of 
corruption such as the protection of monopolies, inequality and injustice. 


The institutional proposal has proven its effectiveness in northern (central) countries, 
and it is not an innovative proposal in Latin America, on the contrary, it has been developed 
by scholars such as Friedman for the case of Chile.*º 


2.2.3 The Institutionalization of Evaluation: Linkage with Citizenship 


The Social Council (SC) is located in the internal evaluation, which has as its 
objectives the monitoring for the rendering of accounts of the administrators of the IESP 
and the academic evaluation that provides information for the continuous improvement of 
the teaching-learning process. 


The social councils are usually formed by citizens (not affiliated with a political 
party or ministers of any religious cult) who are not part of the organization of the Board 
of Directors (Technical Council, University Council, etc.) or of the student community, they 
are generally citizens with a particular interest in improving the efficiency of the school, are 
graduates, retired teachers and researchers, among others. And, the evaluation institutions, 
the social councils and the administrators of the IESPs must generate efforts to improve the 


low performance of University in Mexico. 


It is not enough for educational evaluation organizations to be autonomous.*º If 
citizens do not have mechanisms that inform them about the performance of IESPSs, it will 
be difficult for them to participate actively with their representatives or in civil associations, 
in order to send their demands to those responsible for University. 


On the other hand, it must be noted that these mechanisms must always start from 
the definition of accountability that allows the monitoring of honesty and efficiency in the 
Public Administration, including the evaluation institutions, with the purpose of reducing the 
perceived distrust and uncertainty. by the citizen towards the work of officials. 


38 Cfr. Kent, 2002: 265-274. 

39 ANUIES argues that the evaluation system with civil associations ensures autonomy, since “international experience 
shows that it is not convenient for the evaluation of higher education to fall directly on the State, but rather on indepen- 
dent and intermediate organisms between HEIs and public authorities, with the necessary technical competence and 
social legitimacy. In the most diverse countries, the distinction between autonomous and non-autonomous HEIs is giving 
way to a common status, according to which all HEIs enjoy a wide margin of academic and administrative autonomy 
and, at the same time, all are subject to evaluations. in the spirit of the obligation of accountability to society”. (ANUIES, 
2000: 229 and 230). 
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31 SYNTHESIS OF THE METHODOLOGICAL FOUNDATIONS AND THE 
REQUIRED ATTRIBUTES OF THE EVALUATION OF THE QUALITY OF 
EDUCATION IN UNIVERSITIES PUBLIC OF MEXICO 


3.1 Methodological Foundations of Academic Evaluation in the Public 
Universities of Mexico 

As has been seen throughout the work in progress, it is difficult to propose a single 
methodological process of academic evaluation, since it is possible to consider, rather, 
applying a variety of methodologies depending on the objectives of the evaluation in each 
specific case. Thus, the methodology of academic evaluation, therefore, will be determined 
by its object. 

In addition, to formulate a value judgment about the organization of a teaching 
center or an educational program, it is imperative to compare them with reference instances 
and programs, with what this organization or program would be adequate or inadequate 
compared to the determined references. Thus, the determination of this instance or reference 
program supposes the establishment of evaluation criteria that can be multiple: adequacy, 
desirability of the organization or program, goodness, functionality, effectiveness, etc. 


Once the objectives and criteria have been established, the content to be evaluated 
must be delimited, that is, the aspects of the educational centers or the academic programs 
that must be evaluated must be delimited. And for this, itis necessary to propose a theoretical 
model, which must allow capturing the relationships -in a global way- of all the factors that 
intervene. Thus, the choice of one or another model will condition the evaluation scheme, 
and with it its objectives, its processes and its methodological techniques. 


Regarding the evaluation process, it must be affirmed with category that it 
is conditioned in the first place by the concept of evaluation from which it starts. Thus, 
the methodology followed in the evaluation processes is determined by the models 
and theoretical approaches on which the evaluative objectives are focused, which will 
undoubtedly condition the instruments in the collection of information with its a posteriori 
analysis. 


And, once the objectives have been set, the model has been decided, as well as the 
methodological procedure, itis imperative to collect information, that is, what techniques are 
going to be used to collect and record the information. A large amount of the data necessary 
to evaluate an educational center or academic program is generated through the analysis 
of documents, whether personal, public, formal, informal, etc. Only some of the most 
recurrent techniques will be mentioned, considering that they are extensively developed in 
many manuals. Thus, among the techniques applied in the academic evaluation process, 
the following stand out: questionnaires, institutional documents, interviews, checklists, 


40 And, it must also be mentioned that, given the complexity of educational centers in general, IESPs in particular, as 
well as their academic programs, it is mandatory to resort to very diverse sources of information collection, regardless 
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observation, survey, estimation scales, inventories, on-site visits, discussion groups and 
triangulation, among others. 


In addition, today, in the context of the IESPs in Mexico, the PUs adopt an academic 
evaluation process with a system of reference frameworks, criteria, indicators, standards, 
measurement instruments, incentives, and promotion strategies that for the sake of To 
contribute to improving the quality of teaching and research functions, it takes into account 
a repertoire of componenis (inputs) and processes and results (outputs).*! Within the 
components or inputs are considered: Professors and researchers, students, curricula and 
syllabys (course plans and programs), physical infrastructure and technology (including 
libraries and infotheque), assets and available economic resources, leaders, authorities 
and administrators, and the body of laws, rules and regulations. 


In the processes, the following are considered: The teaching-learning binomial, 
the research and development of technologies, the hiring, promotion and permanence of 
academic and administrative personnel, the designation of authorities and the operation 
of collegiate bodies, the admission, permanence, promotion and certification of students, 
the acquisition, use, operation and maintenance of the physical facilities and technological 
infrastructure, the planning of institutional development (including its evaluation practices), 
the administration and exercise of economic resources, the achievement of economic 
resources and patrimonial assets, and the legislation and adoption of institutional governance 
practices. In addition, the results consider: The institutional climate and image in society, 
coverage in response to demand, learning achieved by students, qualified professionals 
(graduates), registered patents, published work (books, articles, thesis, artistic production) 
and the services provided (technical advice, social assistance, etc.). 


The evaluation ofthe IESPs is their own responsibility, for which they have the capacity 
to carry out their own evaluation on a permanent and daily basis. Such a process constitutes 
the internal evaluation, which needs a complement that is the external evaluation, that is, 
the one that brings together external scrutiny mechanisms. And, as part of this process, the 
CIEES must guide their purposes to the evaluation of the quality of the programs, especially 
those for the training of professionals at the undergraduate level, in addition to evaluating 
the functions of institutional administration and dissemination of culture.? 


of the paradigm chosen one. In other words, in the same way that it is difficult to frame a certain model within a parti- 
cular line, it is also difficult to use only techniques of a certain methodology to deal with educational evaluation as it is 
understood today. 

41 Cfr. Gago Huguet, 2001: 67. 

42 The CIEES program operates based on 9 committees made up of academic peers from different universities that 
evaluate the components (inputs) and the processes of each program of an educational institution (curriculum, profes- 
sors, laboratories, among the main ones). Each committee deals with an area of professions (health, agriculture, engi- 
neering, socio-economics, etc.), or with some general function (administration) of the institution. The committees review 
and analyze the information and elements provided by each program manager, and then carry out visits, receive infor- 
mation, verify and verify. As a result, the committee writes a document with observations, judgments and suggestions, 
which is reviewed with those responsible for the evaluated program before formal delivery and circulation in the pertinent 
instances. From these evaluations, the corresponding actions in charge of each educational center are derived, as well 
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Another aspect of the evaluation is the one that concerns the students and here 
comes the National Evaluation of the Students, which is a set of programs in charge of 
the CENEVAL, an organization that evaluates the extent to which the students achieve the 
learning objectives. of secondary education, baccalaureate or university training (bachelor's 
level) in the professions regulated by the corresponding law, operating based on reference 
profiles and specification tables (knowledge and skills) agreed by technical councils 
(academics, professional associations, service users). The relevant model is the evaluation 
of results based on tests of national official validity (CENEVAL) that provide complementary 
elements to the CIEES to determine the quality of the programs, based on the efficiency 
criterion.*º 


And in addition to the self-assessment and external evaluation schemes, the 
accreditation process has been identified as a mechanism that must guarantee the operation 
of reliable, timely and permanent processes for the improvement of the quality of University. 
Thus, COPAES confers formal recognition in favor of organizations to accredit University 
academic programs offered by both public and private institutions, after assessing their 
organizational, technical, and operational capacity and their evaluation frameworks for 
the accreditation of academic programs., of the administration of its procedures and its 
impartiality. These evaluations for accreditation are based on the Guidelines and General 
Framework for the Accreditation Processes of Higher Level Academic Programs (COPAES, 
1989) established by the council itself.“ And, the scope of its work, is made up of all the 
academic programs grouped in the areas of knowledge of the university higher technical, 
undergraduate and postgraduate levels that are taught by both public and private University 
institutions. 


as elements of judgment for pertinent decisions in external instances such as: Government, associations of schools and 
faculties, or association of professionals. Cf. Gago Huguet: 2001, 81. 

43 The results of these evaluations can affect the admission of students to institutions and in the qualification of profes- 
sionals. 

44 In accordance with these Guidelines, the methodology to be used by the accrediting body must consider: Categories, 
factors or areas; criteria; indicators; and parameters. The categories, factors or areas are those that the accrediting 
body will evaluate in any accreditation process, which must contain at least those proposed by COPAES, and which 
are: Academic staff assigned to the program, curriculum, methods and instruments to evaluate learning, institutional 
services for the learning of students, students, infrastructure and equipment to support the development of the program, 
research lines and activities, if applicable, for the delivery of the program, linkage, institutional regulations that regulate 
the operation of the program, conduction academic-administrative, planning and evaluation process, and administrative 
management and financing. The criteria are axiological references defined a priori, based on which value judgments will 
be issued. In addition, for each category and criterion, the pertinent indicators will be defined, that is, the quantitative and 
qualitative expression of the value of two or more properties of a phenomenon will be specified. And, the parameters, 
standards or benchmarks of evaluation are the ideal or desirable values of an indicator, previously established by the 
accrediting body for each indicator, and which will serve to be contrasted with the indices of the academic program. 

45 The accreditation process is made up of various stages, each of which requires a specific procedure. Firstly, given 
that accreditation is voluntary, the head of the institution must expressly request the accrediting body, recognized by 
COPAES, its intervention to carry out the accreditation process of one or more of its programs, for which it will sign a 
contract. with the same accrediting body. In addition, to start the accreditation process, the programs must meet a set of 
conditions previously established by the accrediting body, in accordance with the General Framework for Accreditation 
Processes for Academic Programs issued by COPAES; however, its compliance will not be a guarantee of accredita- 
tion. Third, the program must carry out a self-assessment that considers, quantitatively and qualitatively, its activities, 
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In summary, it can be noted that the methodological foundations of academic 
evaluation in the UPM, today, are: 1) The methodology of academic evaluation determined by 
its object; 2) the formulation of the objectives and criteria of the evaluation; 3) the theoretical 
model that determines the evaluation; 4) the evaluation methodology is influenced by the 
theoretical models and approaches on which the evaluation objectives are focused; 5) 
the techniques for the collection and recording of information are set by the model and 
methodology implemented; 6) the PUs adopt an academic evaluation process with a system 
of components or inputs (inputs) and processes and results (out-puts); 7) the evaluation 
of the PUs is their own responsibility, for which they have the capacity to carry out their 
own evaluation (internal evaluation), which they complement with the external evaluation; 
8) in addition to the self-assessment and external assessment schemes, the accreditation 
process is proposed as a mechanism that must guarantee the operation of reliable, timely 
and permanent processes for the improvement of the quality of University; and, 9) a complex 
of institutional and intersectoral instances (CONAEVA, CONACYT, ANUIES, FIMPES, 
CENEVAL, CIEES, etc.), has constituted a system of reference frameworks, criteria, 
indicators, standards, measurement instruments and strategies that have Its purpose is to 
qualify the quality of University based on the criterion of efficiency. 


3.2 Formal Foundations of Academic Evaluation in Public Universities of 
Mexico 


In the experience of the academic evaluation of the UPM that has been approached 
in this work, two formal foundations of the evaluation are clearly identified, namely: The 
assurance of educational quality and accountability. 


Regarding the assurance of academic quality, this implies: a) A change in quality 
efforts in schools, which go from focusing on the quality of the teacher towards the study 


organization, inputs, processes, general operation and results, as well as the scope of its objectives and goals. The 
self-assessment must be integrated into a report that will be delivered to the accrediting body. Then, based on the self- 
-assessment report and the evaluation of the conditions to start the accreditation process, the accrediting body will make 
a comprehensive assessment of the program through documentary review and a visit by evaluators to the institution that 
imparts it, where they They will conduct interviews with representatives of sectors of their community and collect additio- 
nal information. The evaluators or visitors must prepare a report of the evaluation carried out, which will contain the pro- 
posal to grant or not the requested accreditation, as well as the recommendations that, if applicable, the program must 
attend to achieve the accreditation. Finally, with the report from the inspectors or evaluators, the accrediting body will 
issue a final opinion which may result in one of the following categories: Accredited (for the program that complies with 
what is established by COPAES regarding quality), non-accredited (given to the program that does not have sufficient 
conditions to be accredited. After a minimum period established by the accrediting body, the program may request your 
accreditation again, provided that you have heeded the recommendations made to you). However, there is a subsequent 
stage to the qualification of the program, and this occurs when the evaluated institution expresses its disagreement with 
the final opinion through a request to the accrediting body, in writing and notifying COPAES, of the review of said final 
opinion. according to the established procedure. If the revised opinion is still unsatisfactory for the institution, an autho- 
rized representative may go to COPAES, which will promote, where appropriate, after analyzing the evaluation opinion, 
the institution's request and the information generated for the accreditation process, a new evaluation process before 
the accrediting body, but now with the presence of experts in the area appointed by COPAES. The evaluation opinion 
will be, in this case, unappealable. 
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and optimization of the institution as a whole; b) the application of new or complementary 
quality control mechanisms in University; and c) The creation for the first time of quality 
assurance systems and mechanisms related to performance within continuous training (Van 
Der Berghe, 1998). 


In addition, according to Cano García (1998), the factors that have the greatest impact 
on educational quality are: Teachers, Curriculum, Evaluation and School Organization. 
Thus, in the first place, the quality centered on the teaching staff has to do with the existence 
of affirmations that make the quality guarantee of the training of its teaching staff depend 
exclusively.“º In a second term, quality focused on training programs consists of planning, 
providing, and evaluating the optimal curriculum (according to the criteria of each country) 
for each student, in the context of a diversity of individuals who learn (Wilson, 1992: 3. 4). 
The quality of the teaching-learning processes is guaranteed when it enables, enhances 
and produces the result of making each and every one of the agents involved more humane. 
And, thirdly, center- and/or institution-centered quality cannot be achieved solely through 
policy measures designed in a standardized and uniform manner. De la Orden points out 
in this regard: “The quality of education, as soon as it is manifested in a valid product, will 
depend fundamentally on what happens in the school, on the structures and processes of 
educational institutions” (De la Orden, 1993: 264). 


In addition, the academic evaluation of the IESP provides accountability -defined as 
the permanent obligation of the leaders or agents to inform their principals or principals of the 
acts they carry out as a result of a delegation of authority that is carried out through a formal 
or informal contract and that implies sanctions in case of non-compliance- the necessary 
tools to monitor the effectiveness and efficiency -political responsibility- of public servants, 
one of two responsibilities towards citizens. Based on accountability, it can be stated that 
IESPs, as well as any other state agency, must meet the definition of accountability. 


The approach to the internal organization of the IESP shows the need for monitoring, 
as well as the design of incentives that guide the interests of the administrator and the 
professor/researcher with the interests of the student/student. Itis not an easy task, however, 
the evaluation, the publication of results and the design of a system of incentives (both 
positive and negative) have demonstrated their efficiency in redirecting tensions within the 
IESPs. The problem lies in the design of the rules that redirect such interests towards the 
community good. The design of an institution for accountability in IPES, must start from the 


46 From this perspective, the responsibility of the educational system is discharged on them and they consider that 
improving the quality of teaching staff requires: rigorous selection procedures that allow only the most qualified and 
highly motivated candidates to be selected; short, practice-based initial pedagogical training; a sufficiently motivating 
remuneration that prevents these professionals from fleeing to other professions; professional development plans and 
an administration that offers possibilities for promotion. Other elements that have been considered important are the 
stability of the teaching staff; teamwork and shared decision making; curricular planning and coordination as well as a 
high degree of autonomy. From this perspective, as Elliot (1993) points out, teacher evaluation can be interpreted as yet 
another strategy of coercive power. From a comprehensive model, educational quality centered on the teaching staff 
must address both the improvement of the teacher and that of the institution. 


Synthesis around the Evaluation of the Quality of Education in the Public Universities of Mexico: Criticism and EE 
Proposal 


laws (legal responsibility) that govern the IPES and the objectives (political responsibility) 
for which they were created. 

In summary, it is possible to note that the formal foundations of academic evaluation 
in the UPM, today, have to do with: 1) quality assurance; 2) accountability; and 3) the 


establishment of an autonomous evaluation institution. 
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CONCLUSIONS 


From the work developed in these spaces, the identification of the following 
methodological foundations of the evaluation of the educational quality in the Public 
Universities of Mexico (UPM) becomes susceptible, namely: 1) The methodology of the 
academic evaluation is determined by the object of it; 2) the formulation of the objectives 
and criteria of the evaluation; 3) the theoretical model that determines the evaluation; 4) 
the evaluation methodology is influenced by the theoretical models and approaches on 
which the evaluation objectives are focused; 5) the techniques for collecting and registering 
the information established by the model and the instrumented methodology; 6) Public 
Universities (UP) adopt an academic evaluation process with a system of components or 
inputs (inputs) and processes and results (out-puts); 7) the evaluation of the PUs is their 
own responsibility, for which they have the capacity to carry out their own evaluation (internal 
evaluation), which they complement with the external evaluation; 8) in addition to the self- 
assessment and external assessment schemes, the accreditation process is proposed as a 
mechanism that must guarantee the operation of reliable, timely and permanent processes 
for the improvement of the quality of University; and, 9) a complex of institutional and 
intersectoral instances (CONAEVA, CONACYT, ANUIES, FIMPES, CENEVAL, CIEES, 
etc.), has constituted a system of reference frameworks, criteria, indicators, standards, 
measurement instruments and strategies that have Its purpose is to qualify the quality of 
University based on the criterion of efficiency. 


Likewise, the document considers the following formal foundations for the evaluation 
of educational quality in the UPM: 1) Quality assurance; 2) accountability; and, 3) the 
establishment of an autonomous evaluation institution. Both types of foundations, the 
methodological and formal, constitute the theoretical foundations of the evaluation of the 
educational quality in the UPM, object of study of the present work. 


Thus, with reference to the foundations outlined and from the theoretical content 
referred to in this essay, the following conclusions can be highlighted: 


First 


The theoretical foundations of the evaluation of educational quality in the UPM guide 
towards the understanding that a quality public university institution is one that is pertinent, 
efficient and effective. And, additionally, it is proposed here that for the evaluation to truly 
serve the purpose of quality improvement, the evaluation practice of each university must 
guarantee utility, equity and flexibility. In addition, evaluation must become an effective 
means of accountability. 


Second 


Conclusions FER 


Asithas been seen throughout the work presented, there is no single quality criterion 
applicable to all University Institutions (HEIs) equally, nor a single model of excellence. In 
the case of the UPM, the previous sentence applies. Faced with reality, either the PSUs 
continue to deceive themselves and the public by offering what they can never fulfill, or 
they begin to accept that their mission is not the same as that of foreign institutions, and 
that they need to look in other directions to find the criteria of excellence that apply to them. 
In other words, the standard of measurement of the evaluation must be what each PSU is 
reasonably capable of achieving given its specific mission and its present reality. Therefore, 
itis reiterated that there cannot be a standard pattern of quality applicable to all UPMs and 
that the evaluation system must be able to handle various criteria of excellence and, with 
this, guarantee an effective evaluative methodological solidity. 


Third 


Based on the above, it is also difficult to propose a single methodological process 
of academic evaluation, since it is possible to consider, rather, applying a variety of 
methodologies depending on the objectives of the evaluation in each specific case. 


Fourth 


The quality assurance strategy, in the case of the UPMs, considers the evaluation 
mechanism, which implies the existence of an organization that is in charge of the process 
of evaluating or accrediting, institutional self-evaluation, evaluation by academic peers, the 
publication of the evaluation report and the transition from evaluation and quality assurance 
processes to accreditation processes. 


Fifth 


The evaluation in the UPM is oriented to the improvement of quality as the main 
objective (formative evaluation). However, the increase in quality is a long-term phenomenon, 
which depends on the will and capacity of the institutions to improve, not on the evaluation 
system. Thus, the evaluation mechanisms that ignore that the quality of University depends 
on the quality of each of the HEIs that comprise it are doomed to failure; and, on the contrary, 
the mechanisms that focus on strengthening the will and capacity of HEIs to improve are the 
only ones that have a chance of achieving that goal. 


Sixth 


In the consideration of the UPM, in the development of this essay the following 
weaknesses in the current evaluation system were identified: 1) An evaluation process 
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without integral participation; 2) evaluation as a political discourse of opposition; 3) the 
absence of a legal framework for the current evaluation system; 4) the lack of transparency 
in the evaluation; 5) the lack of a meta-evaluation; and, 6) evaluation as poor social policy. 


Seventh 


In Mexico, there is no law that contemplates accountability for efficiency in the 
UPMs, except for the internal controls of the Executive, loosely implemented by the SEP. 
Currently, there is an evaluation and accreditation system made up of civil associations, 
which have no obligation to be called to account. Itis suggested, therefore, the formation of 
a State Evaluation Institution with administrative and financial autonomy, as well as political 
independence, which performs academic evaluation as an instrument of accountability.! 
This would prevent their work from being subject to the ups and downs of the political cycle, 
and it is proposed that their financing does not come from the Executive but directly from 
the Income Law. 


Eighth 


In the context of the UPMs, the current evaluation culture is incipient. The particular 
culture of evaluation, which in fact proposes the project of the Executive power of educational 
modernization, does not contribute to making education display all its potential. For this 
reason, the rules must be redesigned to build a project for a democratic country, based on a 
culture of knowledge and participation. The formative evaluation (diagnostic) starts from the 
pedagogical processes and seeks to detect the difficulties and deficiencies of educational 
work in the process itself, so that the modifications are made before the educational 
task ends. Thus, the improvement of the educational system calls for the adoption and 
development of an evaluation paradigm linked to formative evaluation (oriented towards the 
development). 


Ninth 


The current financing policy oriented by academic productivity and implemented by 
the SEP and supported by ANUIES, has produced a redirection of internal tensions that 
harms teaching performance. And it is that, due to the loss of purchasing power, it makes 
teachers compete for 


1 The student in whom the common good is deposited, has the right to participate in a teaching-learning process that 
provides them with knowledge. This knowledge would be recognized through their certificate and/or degree, which in turn 
would be evaluated by the labor market and society in general. Itis in this social-labour assessment that the ultimate goal 
of higher education is found, since it would mean improving the quality of life of the student. IESPs must seek to improve 
their programs or professional profiles, as well as their teaching strategies; however, they must not underestimate all 
those means to spread their scientific and technological culture, since it is the latter that generates the change in the 
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scarce financial resources; and institutions of University -evaluation and accreditation 
of infrastructure to obtain additional federal and state funding, with the student/student being 
the most affected, since their interests are diminished; and as a result, society in general 
ceases to benefit ftom a university culture. 


Tenth 


The proposal of this work involves accountability in the UPM, although it can be 
extended to all universities, including private ones. In addition, the proposal can be useful 
for different levels of education. And the analysis adheres to the constitutional limitations, 
although it is considered to legislate on an organic law that governs the proposed Evaluating 
Institution. The feasibility of the proposal lies in the accountability itself, since it is, in itself, 
a technological solution to the asymmetric information that exists between citizens and 
representatives. However, the lack of interest so far shown by federal legislators in the issue 
of education, and the mistrust of citizens towards the different powers of the State, are the 
main difficulties to overcome. 


Finally, here we express our agreement with Túuennermann (1999: 38) in his utopia 
for the 21st century university, which is one that “(...) accepts evaluation by its peers and 
practices systematic self-assessment of all its activities. In addition, that aware of its 
social responsibility without prejudice, of its autonomy, it recognizes that it is subject to the 
evaluation by society of the efficiency and effectiveness of its performance “accountability”, 
that is, to “rendering of accounts”. This scheme, which is useful today for further reflection, 
may be utopian, but it is worth trying to inspire it in times to come. 
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